This article presents an exploratory study to find out whether high-ability secondary school students in Singapore were able to deal with open mathematical investigative tasks. A class of Secondary One (or Grade 7) students, who had no prior experience with this kind of investigation, were given a paper-and-pencil test consisting of four open tasks. The results show that these students did not even know how to begin, despite sample questions being given in the first two tasks to guide and help them pose their own problems. The main difficulty was the inability to understand the task requirement: what does it mean to investigate? Another issue was the difference between searching for any patterns without a specific problem to solve, and searching for patterns to solve a given problem. The implications of these findings on teaching and on research methodologies that rely on paper-and-pencil test instruments will also be discussed.