In this article, I share my experiences from Week 3, Day 1 of my AI engineering journey. I focus on quantization, a crucial technique that allows the fitting of large models on smaller GPUs, such as a 6GB laptop GPU.
Quantization reduces the precision of the numbers used in model weights, which can significantly decrease the model size and memory requirements. This makes it feasible to fine-tune models that would otherwise be too large for my hardware.
I provide a step-by-step walkthrough of the quantization process, making it accessible for beginners. By the end of this article, readers will have a better understanding of how to implement quantization in their own projects.