Pedestrian traffic is subject to be affected simultaneously by many factors. In order to judge which factor(s) should be taken into consideration when facing a specific pedestrian traffic context for fire safety analysis, it is essential to differentiate the impacts of different factors from each other. This paper aims to develop a method to separate the impacts on pedestrian traffic among different factors. Focusing on an individual pedestrian traffic indicator—walking speed, three important human factors, i.e., gender, age and luggage-carrying were selected to investigate their respective impacts on walking speed. Through video observation in a subway station in Shanghai, nearly 1000 walking speed samples were obtained. These samples include 22 combinations of different levels of age, gender and luggage-carrying. By statistical analysis the impacts of age, gender, and luggage-carrying on walking speed were revealed separately. It is found that except for older pedestrians, males walk faster than females by 5–7%. Middle-aged pedestrians walk slower than the young by 6–8%, and older pedestrians walk slower than the young by 18–24%. Compared with no luggage, small luggage only reduces mean walking speed by 2–3%. For medium luggage, large luggage and trolley cases, the decline rates of walking speeds were 5–8%, 10–14%, and 3–8%, respectively. Research results suggest that special attention and appropriate adjustments such as the modification of evacuation time, speed and capacity of pedestrian flow, should be paid for fire safety analysis in pedestrian traffic sites with noticeable amount of older pedestrians and pedestrians carrying large luggage.