Travel by the elderly is generally characterized by relatively short distances and low frequencies. However, with social development, the travel patterns of the elderly are lacking in sustainability. In some European countries, the elderly are increasingly dependent on cars while traveling. In addition, the influencing factors of the elderly’s travel behavior are also changing. At present, most foreign and domestic studies focus on the impact of individual and household socioeconomic attributes, travel attributes, and policy factors on the travel frequency, travel mode choice, and travel demand of the elderly, but they lack an analysis of the regional differences of the elderly’s travel behavior characteristics and related countermeasures. The studies excessively focus on the influencing factors but overlook the difference between the elderly’s travel characteristics and young people’s travel characteristics, as well as the interactions at the household level. Based on data from the Fifth Travel Survey of Residents in Beijing 2014, this paper uses variance analysis, Spearman’s correlation analysis, and descriptive and comparative analysis to study the difference in travel frequency over 24 hours between the elderly and middle-aged/young people in Beijing, the impact of household, individual, and travel attributes on the travel frequency difference, and the regional difference in the elderly’s travel behavior characteristics. The results show that there is a significant difference in travel frequency between the elderly group and the middle-aged/young group in Beijing; the main reason is the individual difference between travelers. Travelers’ attributes all exert an influence on the travel frequency of both groups, but the degree and direction of the influence are different. At the household level, middle-aged/young people with a higher household income travel less frequently, whereas the case is completely the opposite for the elderly. In terms of personal attributes, gender has a significant negative effect on the elderly’s travel frequency; that is, women travel less than men, whereas there is no difference between men and women in middle-aged/young people. Regarding travel attributes, travel distance and travel duration have a significant negative effect on the two groups’ travel frequencies. The elderly in some European countries are more dependent on cars, whereas a large proportion of elderly people in Beijing walk on foot, but the degree of dependence on cars of the “new generation” of the elderly in Beijing will increase rapidly, which will bring the lack of sustainability of travel patterns, further bringing new challenges to policymakers and transport planning departments.