This paper presents a comparative analysis of rail transit and taxi by travel cost and time based on the large-scale taxi trip data and public transit schedule information in two major U.S. cities. To quantify the relative advantage of one mode over the other, we introduce the notion of travel gradient, which is travel cost difference divided by travel time difference. Based on the signs of travel cost and time differences, we classify all trips into four quadrants. Quadrant II trips are selected for further analysis because rail transit is identified to be competitive with taxi for such trips. We also explore the relation between various trip characteristics and travel gradient with and without considering the spatial variation of such a relation. Main research findings include: (1) around 70% of the taxi trips in the considered datasets can be substituted with rail transit trips if the maximum walking distance is 0.5 miles at each trip end; (2) for around 10% of taxi trips with both modes being viable, rail transit dominates taxi in both travel cost and time; for the rest, rail transit is competitive with taxi; (3) the marginal travel cost saving due to mode switching from taxi to rail transit is about $70; and (4) there exist clearly spatial variations of the relation between trip characteristics and travel gradient. The main policy recommendation from this study is that rail transit can be better marketed by highlighting its relative advantage over taxi in travel time and cost, especially for travels in certain directions and time periods.