Profiling Neural Networks to improve model training and inference speed
Part 1: Learn the basics of performance engineering on neural networks
In a previous post, we studied how to teach the Anki Vector robot to recognize the human sign language. Specifically, we trained a custom Convolutional Neural Network (CNN) with a labelled dataset of 8500 images of human signs taken from Vector’s camera. We demonstrated how the trained CNN can be used to detect human signs in this video. We also explored the tradeoffs between a small custom build CNN model and a large scale well recognized RESNET model. Similar efforts have been made by other researchers; such as an effort to teach Anki Cozmo to learn the human sign language.
In this post, we will figure out ways to optimize this model in terms of the time it takes to train a model (speed of training), and the time it takes to classify an image using this model (speed of inference). We can break down this process into several steps.
Profile your existing model, and find opportunities for improvement.
Make corresponding changes in your model to achieve the desired improvement
Rerun your …