I just downloaded the latest 4 Billion parameter model. I hit ‘Run‘. After a while, the Google Colab instance crashes. Sounds familiar? Well this is bound to happen if we don’t pay attention to the required VRAM and what VRAM we are providing to the model. Quantization is something that can help you tackle this […]
The post Model Quantization Guide: Reduce Model Size 4x with PyTorch appeared first on Analytics Vidhya.
from Analytics Vidhya https://ift.tt/PUsS9Bg
Tags:
Analytics Vidhya