Grokking Generative AI Fundamentals
Vote

0% completed

​

Small, Local and Open Models

A model with 8 billion parameters needs about 16 GB for its weights at 16 bits, as Neural Networks in One Lesson showed. A laptop with 16 GB of memory cannot load them, because the operating system and other apps use part of that memory. At 4 bits, each weight takes half a byte, so the same weights need about 4 GB, and they fit.

At a small law firm, client documents may never leave the office, so a hosted API was not an option

.....

.....

.....

Like the course? Get enrolled and start learning!

Reading Progress

0%

Vote for new content