The COVID-19 pandemic has affected people’s lives throughout the world. Governments have imposed restrictions on business and social activities to reduce the spread of the virus. In the US, the pandemic response has been largely left to state and local governments, resulting in a patchwork of policies that frequently changed. We examine travel behavior across income and race/ethnic groups in Los Angeles County over several stages of the pandemic. We use a difference-in-difference model based on mobile device data to compare mobility patterns before and during the various stages of the pandemic. We find a strong relationship between income/ethnicity and mobility. Residents of low-income and ethnic minority neighborhoods reduced travel less than residents of middle- and high-income neighborhoods during the shelter-in-place order, consistent with having to travel for work or other essential purposes. As public health rules were relaxed and COVID vaccines became available, residents of high-income and White neighborhoods increased travel more than other groups, suggesting more discretionary travel. Our trip purpose model results show that residents of low-income and ethnic minority neighborhoods reduced work and shopping travel less than those of White and high-income neighborhoods during the shelter-in-place order. Results are consistent with higher-income workers more likely being able to work at home than lower-income workers. In contrast, low-income/minorities apparently have more constraints associated with work or household care. The consequence is less capacity to avoid virus risk. Race and socioeconomic disparities are revealed in mobility patterns observed during the COVID-19 pandemic.