This study aimed to determine the characteristics of accelerations and angular velocities obtained by an inertial measurement unit (IMU) attached to a cane between older people with and without physical frailty. Community-dwelling older people walked at a comfortable speed using a cane with a built-in IMU. Physical frailty was assessed using exercise-related items extracted from the Kihon Check List. The efficacy of five machine learning models in distinguishing older people with physical frailty was investigated. This study included 48 older people, of which 24 were frail and 24 were not. Compared with the non-frail participants, the older people with physical frailty had a small root mean square value in the vertical and anteroposterior directions and angular velocity in the anteroposterior direction (p < 0.001, r = 0.36; p < 0.001, r = 0.29; p < 0.001, r = 0.30, respectively) and a large mean power frequency value in the vertical direction (p = 0.042, r = 0.18). The decision tree model could most effectively classify physical frailty, with an accuracy, F1 score, and area under the curve of 78.6%, 91.8%, and 0.81, respectively. The characteristics of IMU-attached cane usage by older adults with physical frailty can be utilized to effectively evaluate and determine physical frailty in their usual environments.