Need help making this model run on my laptop - New Huggingface User
#13
by
programmingdrone
- opened
Hello,
Can someone please guide me as to how I can setup and run this model on my laptop/workstation. I am new Huggingface user and have just about this model while beginning to study AI.
Thanks.
-PG
You can down load a quantized version (gguf) ..
For lower laptops use qk3 version or better qk4
If it does not work then you may find your laptop is restricted to 7b models
Obviously use LM studio as it also tells you what will work and what will not