google

/

gemma-2-2b-it

like 685

Follow

Google 4,515

Hi there! This is a 2.6B parameter model, so storing the weights with bfloat16 precision (like it currently is) would result in a 2.6B parameters * 2 bytes per parameter = ~5.2GB, as seen in the files tab:

When loading your weights, be sure to set torch_dtype=torch.bfloat16, otherwise you will see it take 10GB of RAM/VRAM.

thanks