Physical inactivity is a plague for public health, especially in Western Countries. Among the countermeasures, mobile applications promoting physical activity seem particularly promising, thanks to the spread and adoption of mobile devices. However, the dropout rates of users are high, thereby calling for strategies to increase retention rates. Moreover, user testing can be problematic, because it is typically conducted in a laboratory, leading to a limited ecological validity. In the present research, we developed a custom mobile app to promote physical activity. Three versions of the app were implemented, each featuring a different pattern of gamification elements. Moreover, the app was designed to work as a self-managed experimental platform. A remote field study was conducted to investigate the effectiveness of the different versions of the app. Behavioral log data of physical activity and interaction with the app were collected. Our results show the feasibility of using a mobile app running on personal devices as an independently managed experimental platform. Moreover, we found that gamification elements per se do not ensure higher retention rates, rather it emerged that the richer combination of gamified elements was effective.