Meet NeuralDraw, the AI game app we built in our labs to translate the fun of Google’s Quick, Draw! into a mobile game. We loved the idea of a computer vision algorithm that could guess what you were doodling.
We decided to make it more fun and bring it mobile with the added layer of competition. Download NeuralDraw from the App Store to try it out and let us know what you think. We hope you’ll enjoy it and be the one to out-doodle us all!
NeuralDraw is part of a series of AI app experiments we’re building in Halcyon Labs to showcase the potential of deep learning brought to mobile devices.
What sparked the idea for NeuralDraw
What can you achieve with a neural network that recognises what you’re doodling? The truth is we were really excited with Google’s Quick, Draw! when it launched. We kept playing with it, and checking out other people’s doodles, and throwing around ideas of where smart doodling could be applied: faster sketching? Creative industries? Communications support? Who knows!
Then at one point, we were comparing our doodle skills (what can you do – not all of us are designers) and thinking: man, wouldn’t it be cool if we could actually make a full game out of this? Quickly enough, the ‘what if’ became a ‘why not?’, and that’s how we started working on NeuralDraw. And the one thing we started with was trying to get the technology to work on mobile.
Can you make doodling-based computer vision work on mobile?
Obviously, we made it, or we wouldn’t have anything to write about. Despite the spoilers, let us take you through the journey because building the computer vision behind NeuralDraw was fun.
We built everything from the ground up starting from Google’s dataset gathered through Quick, Draw (big thanks for open-sourcing them and giving us the foundation to work on!).
For NeuralDraw to work, we needed an algorithm that was ‘smart’ enough to be able to tell what you’re doodling, but also quick enough in responding so it didn’t ruin the gameplay.
The AI algorithm we decided on for NeuralDraw
To make things simpler, we decided to switch from the recurrent neural network the Quick, Draw team used, to an on-device convolutional neural network specific for mobiles. They’re different approaches which got the same job done, but we found the later to be better suited for our project needs.
One thing that was lost through the switch of neural networks was the possibility to analyse the sequence of points – the order in which you drew the different lines of a doodle. We found that the algorithm was just as good at guessing the doodle from the completed image, as from the mix of the sequence of points and completed image.
After a few iterations, we ended up with a model that was just 400kB in size. The model does inference in between 14-23ms, depending on the type of device and works even on an iPhone 6 in real time. This was more than enough to allow us to build a fast-paced game on top of it. After all, keeping the algorithm on-device gave it a faster processing speed compared to running it on the cloud. Moreso, we didn’t need to worry about scaling the AI if more people ended up downloading and playing the game.
Shifting from technology to the game experience
Once we were confident we could pull off building NeuralDraw with the right algorithm, we focused on what really sparked this game idea in the first place.
How do we turn Google’s Quick, Draw! Into a fun mobile game? We wanted something you could start playing in a few taps, yet see your progress. That’s why NeuralDraw keeps the already familiar prompt – doodle area combo. The challenge comes from the tournament style scoring system. For each round, you get points that increase your ranking in the game’s leaderboard. Obviously, the more you play, the more points you get. And you can always check your standing in the leaderboard to see if you’re a better doodler than every player of the game.
Who will be the one player to out-doodle them all?
While there’s still a lot of work to be done until we get to see AI-powered doodling taking over the creative world, we’re rooting for you to out-doodle yourselves.
Start playing NeuralDraw and share your feedback!
(available only in the App Store, for now)
We think of this app as an MVP and the beginning of a great game. We’d love to develop it further in many interesting ways, including your input in the process as well.
Review us on the App Store or shoot us a message on your preferred social channel. If you’d like a more in-depth view as a fellow developer tinkering with AI, our team is happy to share their experience and expertise with you. If you think smart doodling could really contribute to an amazing product or additional feature, we’d love to hear your ideas.