Learn With A Robot

Learn With A Robot

Share this post

Learn With A Robot
Learn With A Robot
Profiling Neural Networks to improve model training and inference speed

Profiling Neural Networks to improve model training and inference speed

Part 3: Quantifying the improvements in your model

Amitabha Banerjee's avatar
Amitabha Banerjee
Dec 12, 2021
∙ Paid

Share this post

Learn With A Robot
Learn With A Robot
Profiling Neural Networks to improve model training and inference speed
Share

In this series, we have been exploring how to debug and improve the performance of Convolutional Neural Networks (CNNs). In part 1, we studied how to profile the performance of a neural network using the TensorBoard profiler. In part 2, we examined how to follow up on one of the recommendations from TensorBoard profiler: turning on mixed precision training for our model. In this part we aim to discuss how we can scientifically and methodically quantify the improvements that a change to a CNN model can deliver, and how to conclude whether the same change is desirable or not.

Quantifying model changes

Here are a few principles on how to scientifically quantity model changes:

  1. Make an Apples to apples comparison: Make sure that you thoroughly know and understand what you are comparing. Often there are subtle changes that happen in the background which can cloud any comparisons that we make between two experiments. In the context of this work, it is important to check that the training is performed on the same type of GPU, on the same training dataset, and for the same number of epochs.

  2. Be careful about run to run variations: When we run two experiments, the end result is rarely the same. The differences occur due to many kinds of random and stochastic processes that always happen on a system. Hence, it is important to not judge the results for an experiment from a single run of the experiment. Rather, one should run the experiment multiple times. Results presented from multiple experiments give the readers a much higher degree of confidence.

  3. Use multiple data points to compare and contrast results. Once we have the results from multiple runs of an experiment, we need to use different measures to interpret the results. Often, the mean of the results is used as the sole point of comparison. However, usually mean is not enough. Additional metrics that may be useful for comparison are: i) Standard Deviation, ii) Median, iii) Percentiles: such as 90th percentile.

This post is for paid subscribers

Already a paid subscriber? Sign in
© 2025 Learn With A Robot
Privacy ∙ Terms ∙ Collection notice
Start writingGet the app
Substack is the home for great culture

Share