Program Synthesis Meets Visual What-Comes-Next Puzzles

Abstract

Translate Article

What-Comes-Next (WCN) puzzles challenge us to identify the next figure that "logically follows" a provided sequence of figures. WCN puzzles are a favorite of interviewers and examiners---there is hardly any aptitude test that misses WCN puzzles. In this work, we propose to automatically synthesize WCN puzzles. The key insight to our methodology is that generation of WCN problems can be posed as a program synthesis problem. We design a small yet expressive language, PuzzlerLang, to capture solutions to WCN puzzles. PuzzlerLang is expressive enough to explain almost all human generated WCN puzzles that we collected, and yet, small enough to allow synthesis in a reasonable time. To ensure that the generated puzzles are appealing to humans, we infer a machine learning model to approximate the appeal factor of given WCN puzzle to humans. We use this model within our puzzle synthesizer as an optimization function to generate highly appealing and correct-by-construction WCN puzzles. We implemented our ideas in a tool, PuzzleGen; we found that PuzzleGen is fast, clocking an average time of about 3.4s per puzzle. Further, statistical tests over the responses from a user-study supported that the PuzzleGen generated puzzles were indistinguishable from puzzles created by humans.