vvsotnikov
/

stablelm-tuned-alpha-7b-16bit

Text Generation

text-generation-inference

Inference Endpoints

Model card Files Files and versions Community

vvsotnikov commited on Apr 20, 2023

Commit

1bf767b

•

1 Parent(s): 35ba507

Update README.md

Files changed (1) hide show

README.md +2 -1

README.md CHANGED Viewed

@@ -24,9 +24,10 @@ datasets:
 Get started chatting with `StableLM-Tuned-Alpha 16-bit` by using the following code snippet:
 ```python
 from transformers import AutoModelForCausalLM, AutoTokenizer, StoppingCriteria, StoppingCriteriaList
 tokenizer = AutoTokenizer.from_pretrained("vvsotnikov/stablelm-tuned-alpha-7b-16bit")
-model = AutoModelForCausalLM.from_pretrained("vvsotnikov/stablelm-tuned-alpha-7b-16bit")
 model.cuda()
 class StopOnTokens(StoppingCriteria):
     def __call__(self, input_ids: torch.LongTensor, scores: torch.FloatTensor, **kwargs) -> bool:

 Get started chatting with `StableLM-Tuned-Alpha 16-bit` by using the following code snippet:
 ```python
+import torch
 from transformers import AutoModelForCausalLM, AutoTokenizer, StoppingCriteria, StoppingCriteriaList
 tokenizer = AutoTokenizer.from_pretrained("vvsotnikov/stablelm-tuned-alpha-7b-16bit")
+model = AutoModelForCausalLM.from_pretrained("vvsotnikov/stablelm-tuned-alpha-7b-16bit", torch_dtype=torch.float16)
 model.cuda()
 class StopOnTokens(StoppingCriteria):
     def __call__(self, input_ids: torch.LongTensor, scores: torch.FloatTensor, **kwargs) -> bool: