smpanaro
/

Llama-2-7b-NuGPTQ

Text Generation

text-generation-inference

Inference Endpoints

Model card Files Files and versions Community

smpanaro commited on Mar 27

Commit

94f8ba3

•

1 Parent(s): e0b038d

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -97,7 +97,7 @@ The model is fake quantized which means each weight has <= 16 (2<sup>4</sup>) un
 ```python
 from transformers import AutoModelForCausalLM
-model = AutoModelForCausalLM.from_pretrained("smpanaro/Llama-2-7b-NuGPTQ")
 linear_layers = ["k_proj", "q_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj"]
 count = 0
 for key, tensor in model.state_dict().items():

 ```python
 from transformers import AutoModelForCausalLM
+model = AutoModelForCausalLM.from_pretrained("smpanaro/Llama-2-7b-NuGPTQ", trust_remote_code=True)
 linear_layers = ["k_proj", "q_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj"]
 count = 0
 for key, tensor in model.state_dict().items():