The purpose of this study was to investigate the influence of visual background complexity and task difficulty on players’ performance in a typical video game task, as a function of their expertise. Challenge in a video game is usually defined by the difficulty level of the main task. The use of low-level visual features of game interfaces to modulate the challenge has only been evaluated by a few studies. Two experiments were designed in which action video game players (AVGPs) and non-players of action video games (NVGPs) were engaged in a target-shooting task. The difficulty level of task as well as the complexity of visual background were manipulated. The enhanced attentional abilities of AVGPs allowed them to perform the task better than NVGPs. Both task difficulty and background complexity impaired task performance. However, this impairment was not modulated by player expertise. Recommendations are proposed to game designers to design the challenge around the low-level visual features of game interfaces.