Augmented Reality on Sudoku Puzzle Using Computer Vision and Deep Learning

Abstract

Many people try solving Sudoku puzzles every day. These puzzles are usually found in newspapers, magazines and so on. Whenever a person is unable to solve a puzzle or is running short on time to solve the puzzle, it will be very convenient to show the solved puzzle as an augmented reality. Objectives: In this paper, proposed an optimal way of recognizing a Sudoku puzzle using computer vision and Deep Learning, and solve the puzzle using constraint programming and backtracking algorithm to display the solved puzzle as augmented reality. Also, a comparative performance analysis with the previous work is shown and provided at the end of this paper. Methods: In order to implement augmented reality on to the Sudoku puzzle, image classification itself won’t be sufficient as the solved puzzle has to be shown on top of the area of the unsolved puzzle in the original image. So puzzle detection has to be performed and for doing so proposed work used CNN and Object Localization algorithms. After the detection this should store the values detected in each 9 × 9 cells and ran a constraint programming and backtracking algorithm to solve the puzzle and finally filled the detected empty cells with correct values of the solved puzzle. Applications/Improvements: Usually the Sudoku puzzles that will find in newspapers and magazines are surrounded by a lot of noise such as text (characters) irrelevant to the puzzle and borders of the newspaper which could be similar to a Sudoku puzzle structure. In this paper it emphasize on how to handle such disturbances and improve the performance.