pszemraj commited on
Commit
88e4e66
1 Parent(s): 6e1f2f7

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +9 -3
README.md CHANGED
@@ -23,7 +23,7 @@ This model is a fine-tuned version of [BEE-spoke-data/tFINE-680m-e32-d16-gqa-102
23
  ## testing
24
 
25
 
26
- install [transformers fork with GQA updates for t5](https://github.com/pszemraj/transformers.git) (wip):
27
 
28
  ```sh
29
  pip install -U git+https://github.com/pszemraj/transformers.git@t5-gqa
@@ -36,13 +36,19 @@ then
36
  from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
37
 
38
  tokenizer = AutoTokenizer.from_pretrained("BEE-spoke-data/tFINE-680m-e32-d16-gqa-flan")
39
- model = AutoModelForSeq2SeqLM.from_pretrained("BEE-spoke-data/tFINE-680m-e32-d16-gqa-flan")
 
 
40
 
41
  prompt = "What is the capital of France?"
42
  inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
43
 
44
  generated_ids = model.generate(**inputs, max_new_tokens=64, no_repeat_ngram_size=3)
45
- print(tokenizer.batch_decode(generated_ids, skip_special_tokens=True, clean_up_tokenization_spaces=True)[0])
 
 
 
 
46
  ```
47
 
48
  ## Quick eval
 
23
  ## testing
24
 
25
 
26
+ install [transformers fork with GQA updates for t5](https://github.com/pszemraj/transformers/tree/t5-gqa) (⚠️WIP🚧):
27
 
28
  ```sh
29
  pip install -U git+https://github.com/pszemraj/transformers.git@t5-gqa
 
36
  from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
37
 
38
  tokenizer = AutoTokenizer.from_pretrained("BEE-spoke-data/tFINE-680m-e32-d16-gqa-flan")
39
+ model = AutoModelForSeq2SeqLM.from_pretrained(
40
+ "BEE-spoke-data/tFINE-680m-e32-d16-gqa-flan"
41
+ )
42
 
43
  prompt = "What is the capital of France?"
44
  inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
45
 
46
  generated_ids = model.generate(**inputs, max_new_tokens=64, no_repeat_ngram_size=3)
47
+ print(
48
+ tokenizer.batch_decode(
49
+ generated_ids, skip_special_tokens=True, clean_up_tokenization_spaces=True
50
+ )[0]
51
+ )
52
  ```
53
 
54
  ## Quick eval