Update README.md (#2)
Browse files- Update README.md (c12acd3b731d4262960aa7ee10451ca153ab1559)
- Update README.md (99fc7fd2b6c4808fe987eed26893a7f33974b624)
- Update README.md (3390f6c553dccbea8ad97de6b0d4328bf44893be)
- Update README.md (aec22bcf80e5401dec5f7a20e22cb4d3cf51b503)
- Update README.md (3480bebe65c92860af6620687b2133eeb74e3232)
- Update README.md (342b8976c96c02e25fb9734bcb786c4ea79ad4fb)
- Update README.md (e55ace262fa681a0a7c946e41fdc6663de848020)
- Update README.md (1b5e62aefe5f159160c2efd036d4d2457397f683)
- Update README.md (4a8ebd0e571d5aa56c74e4e41d48573218e36563)
README.md
CHANGED
@@ -296,6 +296,53 @@ Where to send questions or comments about the model Instructions on how to provi
|
|
296 |
|
297 |
**<span style="text-decoration:underline;">Note</span>: Llama 3.1 has been trained on a broader collection of languages than the 8 supported languages. Developers may fine-tune Llama 3.1 models for languages beyond the 8 supported languages provided they comply with the Llama 3.1 Community License and the Acceptable Use Policy and in such cases are responsible for ensuring that any uses of Llama 3.1 in additional languages is done in a safe and responsible manner.
|
298 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
299 |
|
300 |
## Hardware and Software
|
301 |
|
|
|
296 |
|
297 |
**<span style="text-decoration:underline;">Note</span>: Llama 3.1 has been trained on a broader collection of languages than the 8 supported languages. Developers may fine-tune Llama 3.1 models for languages beyond the 8 supported languages provided they comply with the Llama 3.1 Community License and the Acceptable Use Policy and in such cases are responsible for ensuring that any uses of Llama 3.1 in additional languages is done in a safe and responsible manner.
|
298 |
|
299 |
+
## How to use
|
300 |
+
|
301 |
+
This repository contains two versions of Meta-Llama-3.1-8B-Instruct, for use with transformers and with the original `llama` codebase.
|
302 |
+
|
303 |
+
### Use with transformers
|
304 |
+
|
305 |
+
Starting with `transformers >= 4.43.0` onward, you can run conversational inference using the Transformers `pipeline` abstraction or by leveraging the Auto classes with the `generate()` function.
|
306 |
+
|
307 |
+
Make sure to update your transformers installation via `pip install --upgrade transformers`.
|
308 |
+
|
309 |
+
```python
|
310 |
+
import transformers
|
311 |
+
import torch
|
312 |
+
|
313 |
+
model_id = "meta-llama/Meta-Llama-3.1-8B-Instruct"
|
314 |
+
|
315 |
+
pipeline = transformers.pipeline(
|
316 |
+
"text-generation",
|
317 |
+
model=model_id,
|
318 |
+
model_kwargs={"torch_dtype": torch.bfloat16},
|
319 |
+
device_map="auto",
|
320 |
+
)
|
321 |
+
|
322 |
+
messages = [
|
323 |
+
{"role": "system", "content": "You are a pirate chatbot who always responds in pirate speak!"},
|
324 |
+
{"role": "user", "content": "Who are you?"},
|
325 |
+
]
|
326 |
+
|
327 |
+
outputs = pipeline(
|
328 |
+
messages,
|
329 |
+
max_new_tokens=256,
|
330 |
+
eos_token_id=terminators,
|
331 |
+
)
|
332 |
+
print(outputs[0]["generated_text"][-1])
|
333 |
+
```
|
334 |
+
|
335 |
+
Note: You can also find detailed recipes on how to use the model locally, with `torch.compile()`, assisted generations, quantised and more at [`huggingface-llama-recipes`](https://github.com/huggingface/huggingface-llama-recipes)
|
336 |
+
|
337 |
+
### Use with `llama3`
|
338 |
+
|
339 |
+
Please, follow the instructions in the [repository](https://github.com/meta-llama/llama)
|
340 |
+
|
341 |
+
To download Original checkpoints, see the example command below leveraging `huggingface-cli`:
|
342 |
+
|
343 |
+
```
|
344 |
+
huggingface-cli download meta-llama/Meta-Llama-3.1-8B-Instruct --include "original/*" --local-dir Meta-Llama-3.1-8B-Instruct
|
345 |
+
```
|
346 |
|
347 |
## Hardware and Software
|
348 |
|