how to use it? can you share the process, thx
#1
by
suizhuluoqi
- opened
how to use the ckpt for reference
You should clone the original project "https://github.com/THUDM/GLM-130B.git" then set all env done, and read quantization document "https://github.com/THUDM/GLM-130B/blob/main/docs/quantization.md", my huggingface project only provide the result of quant process, in 4 GPU situation.
Change the model config file from configs/model_glm_130b.sh to configs/model_glm_130b_int4.sh in your scripts (e.g. scripts/generate.sh).
Download this project , set the path of the model to the folder of this project.
Add " --from-quantized-checkpoint " when your run scripts just as normal .
Is there a link to use it online?