Building a Chess Detector App on iOS
Machine learning on iOS devices is a topic I’ve wanted to explore for some time. Fortunately now I have a project to do so and the ability to blog about it as I go.
I’m at the beginning stages and think that this journey could be useful to others who want to do something similar, and so I’ll be documenting my progress.
This article is mostly directed towards other developers and technical people, but I think anyone can follow along as long as you google when you encounter anything unfamiliar.
Here’s the problem statement: Detect a chess board from a screenshot, and then determine the position of all the pieces on the chessboard.
I think of it as two related but separate problems:
1) find the chess board
2) and if one is found, determine the position of each piece
Inputs and Outputs
A ML friend told me to think about models as inputs and outputs, and that my job as the human was to be explicit about what the inputs and outputs. Heeding this advice, this problem can be stated as a series of inputs and outputs.
Input — a screenshot that might have a chessboard
Output — coordinates describing the exact location of the chessboard in the screenshot, if one exists
Input — image of a chessboard
Output — 64 individual images, one for each square of the chessboard
Input — chessboard square
Output — piece classification
After the above step, a representation of the board state can be built and then used as input into a chess engine.
Of the three steps above, I’m going to use machine learning for the first and third ones. The second one, slicing the image into 64 individual squares should be trivial as long as the step above it works.
Now that the problem is well defined it’s time to collect and generate data to train the models. That’ll be the next post.