Two critical quality factors for mobile devices (e.g., smartphones, tablets) are battery life and user-perceived performance of User Interface (UI) events, e.g., responsiveness of user action events and frame rate of video playback events. Unfortunately, state-of-the-art solutions have at least one of the following three limitations: 1) they cannot efficiently handle concurrent UI events of multiple apps (either foreground or background) and so may lead to performance imbalance and high energy consumption, 2) they try to regulate UI event performance periodically and thus may not efficiently handle the aperiodicity of user action events, which can result in poor responsiveness or high overheads, and 3) they rely mainly on CPU frequency/voltage scaling and so may have limited energy savings. In this paper, we present SURF, Supervisory control of User-perceived peRFormance, which is designed to overcome the three limitations. First, it dynamically allocates resources to concurrent UI events for balanced performance. Second, SURF uses supervisory control theory to handle the aperiodicity of user action events. Third, it optimizes the allocation of UI events to CPU cores for additional energy savings. SURF features a three-level architecture design that performs the three tasks at different time scales, according to their different overheads and timing requirements. We test SURF on several mobile device models with real-world open-source apps and show that, without causing perceivable performance degradation, it can reduce the CPU energy consumption by 28-98 percent compared to state-of-the-art solutions. In particular, by optimizing the allocation of UI events to the CPU cores, SURF achieves, 28-76 percent more CPU energy savings compared to other solutions.