Physical performance tests aim to assess the physical abilities and mobility skills of individuals for various healthcare purposes. They are often driven by experts and usually performed at their practice, and therefore they are resource-intensive and time-demanding. For tests based on objective measurements (e.g., duration, repetitions), technology can be used to automate them, allowing the patients to perform the test themselves, more frequently and anywhere, while alleviating the expert from supervising the test. The well-known Timed Up and Go (TUG) test, typically used for mobility assessment, is an ideal candidate for automation, as inertial sensors (among others) can be deployed to detect the various movements constituting the test without expert supervision. To move from expert-led testing to self-administered testing, we present a mHealth system capable of automating the TUG test using a pocket-sized smartphone or a wrist smartwatch paired with a smartphone, where data from inertial sensors are used to detect the activities carried out by the patient while performing the test and compute their results in real time. All processing (i.e., data processing, machine learning-based activity inference, results calculation) takes place on the smartphone. The use of both devices to automate the TUG test was evaluated (w.r.t. accuracy, reliability and battery consumption) and mutually compared, and set off with a reference method, obtaining excellent Bland-Altman agreement results and Intraclass Correlation Coefficient reliability. Results also suggest that the smartwatch-based system performs better than the smartphone-based system.