Hugging Face's logo

lunahr
/
SystemGemma2-2b-it

Text Generation
Transformers
Safetensors
gemma2
conversational
text-generation-inference
Inference Endpoints
Model card Files Community
2
Xenova HF staff commited on
Commit
5c17ee9
•
1 Parent(s): 81216ce

Update README.md (#6)

Browse files

- Update README.md (31deb0200f7259f729c631963b81c0c7bacc4fdb)

Files changed (1) hide show
  1. README.md +10 -7
README.md CHANGED
@@ -86,7 +86,7 @@ model = AutoModelForCausalLM.from_pretrained(
86
  input_text = "Write me a poem about Machine Learning."
87
  input_ids = tokenizer(input_text, return_tensors="pt").to("cuda")
88
 
89
- outputs = model.generate(**input_ids)
90
  print(tokenizer.decode(outputs[0]))
91
  ```
92
 
@@ -117,12 +117,13 @@ from transformers import AutoTokenizer, AutoModelForCausalLM
117
  tokenizer = AutoTokenizer.from_pretrained("google/gemma-2-2b-it")
118
  model = AutoModelForCausalLM.from_pretrained(
119
  "google/gemma-2-2b-it",
120
- device_map="auto")
 
121
 
122
  input_text = "Write me a poem about Machine Learning."
123
  input_ids = tokenizer(input_text, return_tensors="pt").to("cuda")
124
 
125
- outputs = model.generate(**input_ids)
126
  print(tokenizer.decode(outputs[0]))
127
  ```
128
 
@@ -139,12 +140,13 @@ quantization_config = BitsAndBytesConfig(load_in_8bit=True)
139
  tokenizer = AutoTokenizer.from_pretrained("google/gemma-2-2b-it")
140
  model = AutoModelForCausalLM.from_pretrained(
141
  "google/gemma-2-2b-it",
142
- quantization_config=quantization_config)
 
143
 
144
  input_text = "Write me a poem about Machine Learning."
145
  input_ids = tokenizer(input_text, return_tensors="pt").to("cuda")
146
 
147
- outputs = model.generate(**input_ids)
148
  print(tokenizer.decode(outputs[0]))
149
  ```
150
 
@@ -159,12 +161,13 @@ quantization_config = BitsAndBytesConfig(load_in_4bit=True)
159
  tokenizer = AutoTokenizer.from_pretrained("google/gemma-2-2b-it")
160
  model = AutoModelForCausalLM.from_pretrained(
161
  "google/gemma-2-2b-it",
162
- quantization_config=quantization_config)
 
163
 
164
  input_text = "Write me a poem about Machine Learning."
165
  input_ids = tokenizer(input_text, return_tensors="pt").to("cuda")
166
 
167
- outputs = model.generate(**input_ids)
168
  print(tokenizer.decode(outputs[0]))
169
  ```
170
 
 
86
  input_text = "Write me a poem about Machine Learning."
87
  input_ids = tokenizer(input_text, return_tensors="pt").to("cuda")
88
 
89
+ outputs = model.generate(**input_ids, max_new_tokens=32)
90
  print(tokenizer.decode(outputs[0]))
91
  ```
92
 
 
117
  tokenizer = AutoTokenizer.from_pretrained("google/gemma-2-2b-it")
118
  model = AutoModelForCausalLM.from_pretrained(
119
  "google/gemma-2-2b-it",
120
+ device_map="auto",
121
+ )
122
 
123
  input_text = "Write me a poem about Machine Learning."
124
  input_ids = tokenizer(input_text, return_tensors="pt").to("cuda")
125
 
126
+ outputs = model.generate(**input_ids, max_new_tokens=32)
127
  print(tokenizer.decode(outputs[0]))
128
  ```
129
 
 
140
  tokenizer = AutoTokenizer.from_pretrained("google/gemma-2-2b-it")
141
  model = AutoModelForCausalLM.from_pretrained(
142
  "google/gemma-2-2b-it",
143
+ quantization_config=quantization_config,
144
+ )
145
 
146
  input_text = "Write me a poem about Machine Learning."
147
  input_ids = tokenizer(input_text, return_tensors="pt").to("cuda")
148
 
149
+ outputs = model.generate(**input_ids, max_new_tokens=32)
150
  print(tokenizer.decode(outputs[0]))
151
  ```
152
 
 
161
  tokenizer = AutoTokenizer.from_pretrained("google/gemma-2-2b-it")
162
  model = AutoModelForCausalLM.from_pretrained(
163
  "google/gemma-2-2b-it",
164
+ quantization_config=quantization_config,
165
+ )
166
 
167
  input_text = "Write me a poem about Machine Learning."
168
  input_ids = tokenizer(input_text, return_tensors="pt").to("cuda")
169
 
170
+ outputs = model.generate(**input_ids, max_new_tokens=32)
171
  print(tokenizer.decode(outputs[0]))
172
  ```
173