Transfer Learning (TL) in Machine Learning is a technique where we use the learning of a model trained on a different dataset to then adapt and retrain on our own dataset. TL is one of the main ways by which one could make ML successful, specially in situations where the dataset that we have access to is limited.

How does Transfer Learning work?

Lets consider how our human body works. When we learn one skill such as riding a bicycle, we find it much easier to learn related skills which involve balancing, such as learning how to skate, or use roller blades, or ride a scooter.

Training a ML model works in similar ways. If we train a ML model from scratch, we start with random values for the parameters of the ML model, and then start a process known as stochastic gradient descent to reach optimal values. In the case of transfer learning, we would start with values acquired from a previous training exercise instead of picking completely random values. This often leads to a more optimal model, because there is more chance that the model would converge to an optimum state during the training exercise.

When is Transfer Learning useful?

Transfer learning is very useful when we have a limited dataset. As an example, in our exercise of having a Vector robot detect another Vector, our labelled dataset comprises of 685 images, which is a very low count for training a supervised ML model. On the other hand, there are far richer datasets such as the COCO dataset. Thus, starting from a model trained on a rich dataset ensures that we have a good start point, and the main challenge is to optimize to achieve our specific task of recognizing another Vector instead of images in the COCO dataset.

See it in practice…

The following tweet has an example of how Vector could recognize another Vector with A) A model built from scratch without transfer learning, and B) A model built with transfer learning. We have a poll open for the next 24 hours. It would be fun if you could see both videos and rate which one you like better. This is transfer learning in practice, and could be a great learning demonstration.

Thanks again and look forward to see the polls.

Amitabha @hiamitabha

Does Transfer learning (TL) help? Here is an experiment with the TL training service @roboflow. We desire the Vector robot to detect another Vector. Here are 2 videos: A)Without TL B)With TL A) youtu.be/4r8WgqLFy_8 B) youtu.be/YZepFR28_Ww Which result do u think is better?Vector Recognizes Vector - ML model built with transfer learningWe trained a ML model at Roboflow... this time using transfer learning from a model trained with the public COCO dataset. Then used it to let one Vector robo...youtu.be

February 12th 2022