PROTOCOL: Evidence and gap map: studies of the effectiveness of transport sector interventions in low- and middle-income countries.

Abstract

Context: In the Footsteps of Mr. Kurtz Michela Wrong describes walking down the overgrown disused railway which years before had been part of a network linking DRC's copper mines to ports in Angola and South Africa. Despite new investments in the last decade—the Benguela Railway link from DRC to Angola reopened in 2018 after being closed for 34 years22 https://www.railjournal.com/africa/angola-drc-rail-link-restored-after-34-years/. —Africa's rail system is small compared to that in other parts of the world, and a substantial part of what there is not used (Bullock, 2009). The poor state of railway transport in Africa—and the unrealised potential of inland waterways—puts excess pressure on the fragile road transport system, so that transport costs—which are increased by uncompetitive practices—are a break on African development. While much of Africa is an extreme case, inadequate transport infrastructure is an issue across much of the developing world. There are great disparities in the quantity and quality of infrastructure. European countries such as Denmark, Germany, Switzerland, and the UK have close to 200 km of road per 100 km2, and the Netherlands over 300 km per 100 km2. By contrast, Kenya and Indonesia have <30, Laos and Morocco <20, Tanzania and Bolivia <10, and Mauritania only 1 km per 100 km2.33 https://knoema.com/atlas/ranks/Road-density. As these figures show, there is a significant backlog of transport infrastructure investment in both rural and urban areas, especially in sub-Saharan Africa (Foster & Briceño-Garmendia, 2010). The situation is often exacerbated by weak governance and an inadequate regulatory framework with poor enforcement which lead to high costs and defective construction. The wellbeing of many poor people is constrained by lack of transport, which is called “transport poverty.” Lucas et al. (2016) suggest that up to 90% of the world's population are transport poor when defined as meeting at least one of the following criteria: (1) lack of available suitable transport, (2) lack of transport to necessary destinations, (3) cost of necessary transport puts household below income poverty line, (4) excessive travel time, or (5) travel conditions unsafe or unhealthy. Benefits of better transport: better transport policies, infrastructure and services are widely believed to be important to boost sustainable, inclusive growth in low- and middle-income countries (LMICs) in other regions (see, e.g., Berg et al., 2017; Abdul Quium, 2019; Simon, 2002). Transport allows people to reach jobs, education, markets, social services and engage in social and political life. Sustaining rapid economic and social development in LMICs presents a range of challenges for the transport system, a central one being to provide the capacity to accommodate increased volumes of passenger and freight traffic (Simon, 2002). Production: Transport investments can transform economies by supporting structural change, notably the shift of the population from agriculture to manufacturing and services (e.g., Calderon, 2009; Kodongo & Ojah, 2016). A study of rural roads in Bangladesh found they reduced poverty through higher agricultural production, lower input and transportation costs, and higher agricultural output prices at local village markets, as well as increasing secondary school enrolment (Khandker et al., 2009). Incorporating transaction costs into a computable general equilibrium model of Uganda, Gollin and Rogerson (2010) show that better infrastructure will stimulate agricultural production through higher farmgate prices. Consumption and prices: Better transport can make commodities more easily available and affordable. For example, the expansion of railways across India from the 1850s enabled market integration, which reduced prices of basic commodities such as salt.44 A brief overview of studies is given in https://blogs.lse.ac.uk/southasia/2013/04/29/railways-and-indian-economic-development/. Transport-induces changes in location of production and habitation (i.e., changes in land use) and so will affect land values. Deng et al. (2008) show that the increasing density of highways in China is a significant factor driving urban land expansion. Chalermpong (2007) estimates an elasticity of residential property prices with respect to distance from rail transit stations of −0.09. Access to services: Many studies show that long travel times, lack of transport services and high transportation costs are barriers to use of health services; for example, the systematic review (SR) by Kyei-Nimakoh et al. (2017) of 160 studies of barriers to obstetric care in sub-Saharan Africa. These benefits are more fully elaborated in the theory of change below: These benefits may not be realised, or be partly undermined, by the political economy context and the governance framework (Flyvbjerg, 2005; Klopp, 2012; and Alexeeva et al., 2008). Corruption and restrictive practices drive up costs, and public private partnerships (PPPs) often end up costing more than planned (Fatokun et al., 2015; Guasch et al., 2014). Transport costs are high in sub-Saharan Africa even when the road infrastructure is adequate, due to a of lack of competition. Such considerations are an important part of the overall policy framework (Hine, 2014), but beyond the scope of this map, which is concerned with studies of effectiveness, that is, the difference transport makes to outcomes of interest. It is thus argued that better transport is a key component to achieving several Sustainable Development Goals (SDGs): “There are a number of SDG targets directly linked to transport, including SDG 3 on health (increased road safety), SDG 7 on energy, SDG 8 on decent work and economic growth, SDG 9 on resilient infrastructure, SDG 11 on sustainable cities (access to transport and expanded public transport), SDG 12 on sustainable consumption and production (ending fossil fuel subsidies) and SDG 14 on oceans, seas and marine resources. In addition, sustainable transport will enable the implementation of nearly all the SDGs through inter-linkage impacts. Access to sustainable transport for all should be at the forefront, including for vulnerable groups such as women, children, persons with disabilities and the elderly.”55 https://sustainabledevelopment.un.org/index.php?page=view&type=20000&nr=802&menu=2993. However, the presence and extent of these benefits depends on context: there is a great difference between those living in remote rural areas with little contact with the outside world and residents of a slum next to a highway in a rapidly growing city. How they interact with, transport services and policies, however, varies greatly. The impact of transport also depends on factors such as employment opportunities, access to markets and distribution of health and education facilities and other factors which may affect use of all of these. The map has to capture this full range of relevant interventions and possible policies, as well as the possible harms which may arise from transport. Possible adverse consequences of infrastructure investments: Transport can bring disadvantages to some: displacement to make way for construction, poor road safety, higher land prices, spreading disease, air pollution, reduced accessibility on foot, moving access to jobs and goods further away and adverse cultural effects. While transport infrastructure and services generally improve access to social services, they may have adverse effects on both health and education through the role of transport in spreading disease (the Black Death, HIV/AIDS in Africa in the 1980s and 90s, and COVID-19 in 2020—see, e.g., Apostolopoulos & Sonmez, 2006), accidents, and a busy road through a village stopping parents sending young children to school (Jeyaranjan et al., 2010). Over 80% of road traffic deaths occur in developing countries (WHO, 2018). Some these factors are not captured in most analyses, so there is a risk that, if adverse effects are not measured, the cost effectiveness of transport investments could be overstated and they may not produce the full range expected benefits, hence the importance of the regulatory framework. Understanding how transport policies can produce growth-inducing effects and have social benefits, while taking into account possible adverse effects, can guide setting priorities in the strategic use of scarce resources, and setting the regulatory framework for transport investments. The challenge for transport development is thus to realise the benefits while minimizing the adverse consequences. The scope covers: (1) types of transport; (2) the policies and other actions to promote transport-related development; (3) the outcomes of interest; (4) the population of interest; and (5) eligible study designs. Outcomes and study designs are discussed below. Here we specify (1), (2) and (4). The map will include interventions related to all kinds of transport: rail/tram, road and on foot or Motorbike/Bicycle by land, both inland waterways and international maritime transport, and air. Road transport is the fastest surface mode of transport door-to-door and is most suited to short- to medium-distance haul traffic. Roads provide the flexibility, the ability to provide door-to-door service, while providing interchange terminals for rail, water, and air transport. In many LMICs, road transport is the most dominant mode of motorised transport. For example, in the Nile region of Egypt, road transport accounts for 80% of the goods and 90% of the motorised passenger traffic in the region (Nile Basin Initiative, 2012, p. 190). Walking is the most common means of transport in many countries, and bicycles are common for both personal and commercial use. However, the map is not about the use of a transport modality per se, but interventions which affect those choices. So a study of the effects of construction of foot paths, cycle lanes and foot bridges is included, but a study comparing say travel times or health benefits of walking, cycling and driving is not. Railways are the most cost-effective mode of transport for moving bulk cargo for long distances over land and are well suited to container traffic between ports and capitals. Studies have shown that rail transport costs are about 50% lower than road transport costs (Berg et al., 2017). Trams in urban areas are one of the main methods of mass passenger transit though their use has reduced considerably in the last few decades and is not common in many cities across LMICs, most notably but not only in sub-Saharan Africa. Maritime transport is the most dominant mode of transport for moving freight between countries with ports across the global market. Transport over sea has significant cost advantage over surface transport for dry and liquid bulk cargoes or containerised cargo but can be considerably slower than the alternatives. Where there is a steady flow of nonperishable products then this time factor matters less. Maritime transport is important as a transit route for international trade. Inland water transport on the other hand, has the advantage of being cheap, energy efficient, relatively safe, and environmentally friendly. The main type of goods and services using this transport mode comprise of agricultural produce, livestock, fish, general merchandise, and passengers. Additionally, inland ports, linked to other modes of transport connect to international markets, and handle export and import traffic of agricultural products and manufactured goods. But inland waterways are underdeveloped in LMICs, and virtually absent in sub-Saharan Africa (International Navigation Association, 2009). Air-transport is the fastest mode of transport and is best suited for long-distance movement of passengers, perishable products and high value, low-income/low-weight products. Air transport is linked to transport of perishable and frozen foods and precious metals. It is also important in transporting migrant labour, for example, workers from Africa and South Asia working in Gulf countries. Infrastructure investments cover both new infrastructure and maintenance of existing infrastructure. Infrastructure investments entail building or maintaining new transport infrastructure (e.g., roads, railways, ports, or airports), upgrading existing links and technology, or improving transport services, such as public bus services. Information and incentives which cover a range of behaviour change interventions, which would include information (including training) for road safety, and tariffs (prices) for all forms of transport including road pricing and taxes intended to affect transport use, for example, air fuel taxes. Information includes public safety campaigns around speeding, seat belt use, and so on. Price incentives include subsidies or taxes to influence mode choice and transport behaviour (e.g., student fare reductions, tolls, parking fares, fuel taxes, and clean transport subsidies). Institutional framework which is broadly defined to include all policies and the regulations (in the case of overlap with incentives then the policy is classified as incentives). Regulations are rules to directly reduce emissions (such as fuel emission standards or driving restrictions) or to organise the transport sector including ensuring it is competitive (e.g., freight, taxis, or buses) or the construction of infrastructure. The population are all those in all LMICs—as defined by the World Bank—in both rural and urban areas. Both national and international transport are included, but international only insofar as it affects outcomes in LMICs. Conceptual framework of the EGM: how are the interventions expected to work. Several sources present theories of change figured for transport interventions, for example, Berg et al. (2017), Raitzer et al. (2019), and Abdul Quium (2019). Our theory of change, shown in Figure 1, draws on each of these to give a high-level representation which applies to all our included modes of transport. The high-level approach, cuts across all modes of transport, emphasises that there are some common causal pathways for the different modes, meaning that there are likely to be common lessons across sectors which may get overlooked by researchers and policymakers specialised in just one sector. The theory of change shows the causal chains through which inputs are turned into outputs, intermediate and final outcomes, and higher order welfare effects (impact). On the left of the figure are the intervention areas of investment and maintenance, information and incentives, and the institutional framework (policies and regulations). As mentioned above, these effects are mediated by the political economy context and governance framework. Economic and social equity and development: Effects on both economic development through trade, productivity and growth, and social development in various forms through better access. Adverse effects on displaced populations who lose their land or livelihood will also be captured here. Transport planning may mean that transport makes life harder for the poor note easier if the way in which they travel is marginalised, such as roads without pedestrian access. Cultural effects: The positive and negative consequences of increased mobility within and between nations. The increased mobility of the population may have effects on the culture beliefs, values, customs and norms. An example is cultural heterogeneity resulting from migration to urban areas which can result in the loss of traditional values. Health: Health is separated out as there are many channels through which transport can affect health, both positive (access to health services, higher income, availability of more diversified diet, etc.) and negative (road traffic injuries, air pollution, and spreading disease). Sustainability: Transport can have adverse effects on the environment, through impact on land use and local flora and fauna. Congestion is a growing problem, contributing to air pollution from increased traffic volumes. This framework is used to define the categories of interventions and the outcomes along the causal chain to be shown in the map. Why it is important to develop the EGM Although there is no separate SDG for transport, of the 17 SDGs, seven (Goals 2, 3, 7, 9, 11, 12, and 13) include one or more targets that addresses transport, both rural and urban; and 4 (Goals 2, 3, 9, and 11) make specific reference to transport and infrastructure (United Nations, 2016). According to the Institute of Transportation and Development Policy, “this elevation of transport in SDGs recognises it as a key tool in reducing emissions, improving equity, and reducing poverty.” Analysis of these goals identifies the following key aspects of transport in the SDGs: access (urban, rural, affordable for all), road safety, fuel type/efficiency; quality, reliable, resilient, and sustainable infrastructure; regional and trans-border transport; sustainable urban transport for all; reduce vehicle emissions/air pollution in cities; reform fossil-fuel subsidies; rural/urban logistics, supply chain efficiency; and mitigation and adaption of climate change. The literature on the impact of transport policies covers a variety of interventions and outcomes at different levels, such as micro, meso and macro. Due to the wide variety of interventions, mechanisms, and outcomes, a simple way to formalise the impact of transport policies is to how these policies affect the welfare of individuals or groups, improve regulation and infrastructure, would be quite useful. At the same time, as explained above, the expansion of transport in LMICs has brought out both positive and negative effects. The purpose of this map is to document all relevant studies, from all sectors, which analyse the effects of transport interventions. The nearest study to what we will do is the ADB review of transport impact evaluations by Raitzer et al. (2019). But that review was not systematic and more restricted to analysis by economists. We have a broader disciplinary scope, most notably bringing in the relevant engineering literature. Existing EGMs and/or relevant SRs A map of evidence maps conducted in LMICs identified no EGM conducted around transportation (Phillips et al., 2017). The lack of such a map was the rationale for starting this map. There is an on-going global map of road safety (Mohan et al., 2020). Table 1 lists some reviews of transport sector interventions. These are illustrative of the sort of topics, which may be covered; they have not been screened to determine whether they include primary studies from LMICs. The aim of the EGM is to identify, map and describe existing evidence on the effects of transport sector interventions related to all means of transport (roads, paths, cycle lanes, bridges, railways, ports, shipping and inland waterways, and air transport). These interventions are classified as shown in the theory of change (Figure 1), that is infrastructure and maintenance, information and incentives, and institutional framework (including regulations) transportation in LMICs. The primary outcomes of this EGM are also shown in Figure 1 and include transport infrastructure, economic and social development, safety, environmental and environmental health, and economic and equity outcomes. Develop a clear framework of interventions and outcomes related to effects of transport in LMICs Map available SRs and primary studies of the social and economic effects of interventions aimed at improving transportation in LMICs in this framework, with an overview provided in a summary report. Provide database entries of included studies, which summarise the intervention, context, study design, and main findings. The output of the project will be an online, interactive evidence and gap map (EGM) for all forms of transport, such as roads, railways (including mass transportation and bulk transport of energy and related commodities), civil aviation, ports and inland water transport, and urban transport. While SRs aim to identify, assess and summarise research findings from studies on a (narrow) research question, the objective of EGMs is to provide a picture of the completeness of existing research literature on a given topic. As such, EGMs have a broader scope than SRs, and SRs go further than EGMs in processing the contents of the identified research. Another important difference between EGMs and SRs is how they are disseminated. SRs are disseminated as research reports or journal articles, where the answer to the research question is the key issue for readers. EGMs can also be disseminated as a report or an article, but the more user-friendly EGMs display its results in an interactive matrix. Identified studies are plotted in the matrix, so that the user can find evidence, or lack thereof, for his or her particular topic of interest, at a glance. EGM are global public goods that attempt to democratise high quality research evidence for policy makers, practitioners, and public and research funders. The EGM presented here includes evidence from impact evaluations and SRs. The target population are populations living in LMICs. Populations subgroups of interest, which are informed by PROGRESS+ equity criteria (O'Neill et al., 2013) include: rural/urban, women, disability, older population, ethnicity, poorer populations, region, and country. These subgroups will be added to the map as filters. The EGM will include any intervention aiming to construct, improve, maintain or affect the use of transportation in LMICs in the above categories of modes of transport. Broadly, there are three policies that have contributed to improving transport networks; infrastructure investments, price instruments (which we broaden to incentives), and regulations (Berg et al., 2017). The infrastructure investments entail building new transport infrastructure (e.g., roads, railways, ports, or airports), upgrading existing links and technology, or improving transport services. The incentives include subsidies or taxes to influence mode choice and transport behaviour (e.g., student fare reductions, tolls, parking fares, fuel taxes, and clean transport subsidies). The regulations include rules to directly reduce emissions (such as fuel emission standards or driving restrictions) or to organise the transport sector (e.g., freight, taxis, or buses) or the construction of infrastructure. Some policy interventions may affect supply, such as infrastructure investments, whereas others target demand, as do subsidies for transport. We reframe Berg et al.'s three categories (infrastructure, prices, and regulations) a bit more broadly as infrastructure, incentives, and institutions. So, the intervention categories are each mode of transport, and the subcategories in each case those just mentioned, that is, infrastructure, information, and incentives (which is broader than price mechanism) and institutions (which is broader than regulation). Table 2 shows the resulting set of intervention categories. The outcomes are listed in outcome domains ordered along the causal chain (Table 3). Each domain has a number of subdomains. The map covers positive and adverse outcomes, and sufficient scope to capture unintended outcomes. While, cost benefit/benefit-cost analysis is an exercise to determine the social welfare effects of transport sector interventions in comparison to costs, economic impact analysis is an exercise to determine how a transport intervention project or policy affects the amount and type of economic activity in a region. In addition, if included studies report costs related to the costs of transport infrastructure, their cost-effectiveness or cost-benefits, and/or economic impact and/or transport justice/equity these will be reported as well. There are many policy-relevant areas of research on transport, including barriers to access, costs and governance arrangements. Qualitative data and studies can play an important role on complementing impact evaluations; see White (2011) on mixed methods impact evaluations in infrastructure. However, this transport map is a map of effectiveness studies, and so excludes qualitative studies. The rationale is the comparative lack of measures of impact on outcomes of interest using impact evaluation methods. But there is a growing literature. Making this literature discoverable and accessible will be the main contribution of this map. The map is timely because the number of impact evaluations has been growing across development sectors. By impact evaluations we mean studies which assess the difference an intervention makes to outcomes, employing a technique which handles the possible endogeneity of exposure to the intervention. This endogeneity is at the heart of discussions on transport and development. In the Handbook of Transport and Development (in which the cases are mostly from developed countries), the authors state in the introduction that “Often it seems that development follows the transport infrastructure… But the causality is rarely in one direction and often the development form helps shape the transport infrastructure investments” (Hickman et al., 2015, p. 3). This EGM will include impact evaluations and SRs of the effectiveness of transport sector interventions. Experimental designs: randomized controlled trials (RCTs) and natural experiments Nonexperimental designs: (i) quasi experimental designs using statistical methods to create a comparison group such as propensity score matching and regression discontinuity, (ii) regression-based designs such as instrumental variables and Heckmann sample selection models; and (iii) other studies with a comparison group. Before versus after studies with no comparison group will not be included. Regression designs which control for confounding variables. We will not include qualitative research. All included impact evaluations must have been conducted in LMICs as defined by the World Bank. SRs containing evidence only from high-income countries will be excluded. For civil aviation, we will exclude studies focusing on passenger transport. We will search for and include completed and on-going studies. We will not exclude any studies based on language or publication status or publication date. We will use the following strategies to identify completed and on-going potential studies: Database: EconLit, Embase (Ovid), MEDLINE (Ovid), CAB Global Health, CAB Abstracts, Cochrane Library, ERIC (Proquest), Social Sciences Citation Index (Web of Science), Transport Database (Ovid) (or https://trid.trb.org/), Proquest Dissertations & Theses, WHO Global Health Library, Ebsco Discovery. Organisational searches will include: 3ie impact evaluation repository, Asian Development Bank (ADB), African Development Bank, Inter-American Development Bank, Department of International Development (DFID), US Agency for International Development, World Bank (DIME, Impact Evaluations), IFPRI, IPA, Transport Research Institute, Napier University UK, European Conference of Transport Research Institutes, International Rail Research Board (IRRB), Asian Institute of Transport Development, Institute of Transport Research, European Conference of Transport Research Institutes. We will contact a selection of these agencies for details of on-going research. Bibliographic searches: We will screen the SRs to locate additional primary studies. We will also conduct bibliographic back-referencing of reference lists of all included SRs to identify additional primary studies and SRs. In addition, we will identify developing country studies in the on-going map of road safety interventions. In the case of on-going studies we will search for the completed study and include the study as on-going if no completed study is found. Hand search of journals: we will identify key journals from the search results and hand search the contents of the last five years of up to 12 of these journals. Appendix A presents an example of the search strings used for publication databases and search engines, with terms for interventions, regions, and methodologies. We will use EPPI reviewer to assess studies for inclusion at both the title/abstract and full-text screening stages. Two researchers will screen each title/abstract and each full-text. Any disagreements on inclusion will be resolved through discussion. For impact evaluations, we will use a standardised data extraction form to extract descriptive data from all studies meeting our inclusion criteria. Data extracted from each study will include bibliographic details, intervention types and descriptions, outcome types and descriptions, study design, context/geographical information, details on the comparison group, and implementation details. A full list of data to be extracted is described in the coding tool in Annexure C (Data extraction template); this tool will be piloted to ensure consistency in coding and resolve any issues or ambiguities. Two researchers will conduct the data extraction for each study; however, all coders will be trained on the tool before starting and a sample will be double coded to check for consistency. For SRs, a modified version of the tool will be developed for the data extraction. All SRs will be appraised for quality using the AMSTAR2 or ROBIS tool. Critical appraisal will be completed separately by two team members. We will not be critically appraising the quality of the included impact evaluations but will collect data on study design. Where multiple papers exist on the same study (e.g., working paper and a published version), the most recent open access version will be included in the EGM. If the versions report on different outcomes, an older version will be included for the outcomes not covered in later versions. The matrix and filters are described in Appendix A (Evidence matrix). In brief, the matrix will display intervent