Nowadays, mobile devices have become the prevailing computation platform. Due to their battery-constraint nature, delivering excellent user experience with low energy consumption is highly desirable for mobile devices. Past researches on optimizing the energy efficiency for mobile devices are at either too high or too low levels. The former can explore different QoS requirements for applications, while the latter can model the interaction details. To get both advantages, we target a level in the middle, i.e., user behavior level. In this work, we characterize user behaviors, analyze their QoS requirements and propose corresponding QoS models for each behavior type. Then, we introduce U-ACT, a user behavior aware power management framework, which can optimize CPU frequencies for all different behavior types. Our evaluation shows the proposed framework can achieve up to 62% combined improvement, i.e., the summation of QoS and energy improvements in percentage, comparing with the state-of-the-art governors.