reach-vb HF staff commited on
Commit
50ba44f
1 Parent(s): 6eaee04

Update README.md (#2)

Browse files

- Update README.md (c12acd3b731d4262960aa7ee10451ca153ab1559)
- Update README.md (99fc7fd2b6c4808fe987eed26893a7f33974b624)
- Update README.md (3390f6c553dccbea8ad97de6b0d4328bf44893be)
- Update README.md (aec22bcf80e5401dec5f7a20e22cb4d3cf51b503)
- Update README.md (3480bebe65c92860af6620687b2133eeb74e3232)
- Update README.md (342b8976c96c02e25fb9734bcb786c4ea79ad4fb)
- Update README.md (e55ace262fa681a0a7c946e41fdc6663de848020)
- Update README.md (1b5e62aefe5f159160c2efd036d4d2457397f683)
- Update README.md (4a8ebd0e571d5aa56c74e4e41d48573218e36563)

Files changed (1) hide show
  1. README.md +47 -0
README.md CHANGED
@@ -296,6 +296,53 @@ Where to send questions or comments about the model Instructions on how to provi
296
 
297
  **<span style="text-decoration:underline;">Note</span>: Llama 3.1 has been trained on a broader collection of languages than the 8 supported languages. Developers may fine-tune Llama 3.1 models for languages beyond the 8 supported languages provided they comply with the Llama 3.1 Community License and the Acceptable Use Policy and in such cases are responsible for ensuring that any uses of Llama 3.1 in additional languages is done in a safe and responsible manner.
298
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
299
 
300
  ## Hardware and Software
301
 
 
296
 
297
  **<span style="text-decoration:underline;">Note</span>: Llama 3.1 has been trained on a broader collection of languages than the 8 supported languages. Developers may fine-tune Llama 3.1 models for languages beyond the 8 supported languages provided they comply with the Llama 3.1 Community License and the Acceptable Use Policy and in such cases are responsible for ensuring that any uses of Llama 3.1 in additional languages is done in a safe and responsible manner.
298
 
299
+ ## How to use
300
+
301
+ This repository contains two versions of Meta-Llama-3.1-8B-Instruct, for use with transformers and with the original `llama` codebase.
302
+
303
+ ### Use with transformers
304
+
305
+ Starting with `transformers >= 4.43.0` onward, you can run conversational inference using the Transformers `pipeline` abstraction or by leveraging the Auto classes with the `generate()` function.
306
+
307
+ Make sure to update your transformers installation via `pip install --upgrade transformers`.
308
+
309
+ ```python
310
+ import transformers
311
+ import torch
312
+
313
+ model_id = "meta-llama/Meta-Llama-3.1-8B-Instruct"
314
+
315
+ pipeline = transformers.pipeline(
316
+ "text-generation",
317
+ model=model_id,
318
+ model_kwargs={"torch_dtype": torch.bfloat16},
319
+ device_map="auto",
320
+ )
321
+
322
+ messages = [
323
+ {"role": "system", "content": "You are a pirate chatbot who always responds in pirate speak!"},
324
+ {"role": "user", "content": "Who are you?"},
325
+ ]
326
+
327
+ outputs = pipeline(
328
+ messages,
329
+ max_new_tokens=256,
330
+ eos_token_id=terminators,
331
+ )
332
+ print(outputs[0]["generated_text"][-1])
333
+ ```
334
+
335
+ Note: You can also find detailed recipes on how to use the model locally, with `torch.compile()`, assisted generations, quantised and more at [`huggingface-llama-recipes`](https://github.com/huggingface/huggingface-llama-recipes)
336
+
337
+ ### Use with `llama3`
338
+
339
+ Please, follow the instructions in the [repository](https://github.com/meta-llama/llama)
340
+
341
+ To download Original checkpoints, see the example command below leveraging `huggingface-cli`:
342
+
343
+ ```
344
+ huggingface-cli download meta-llama/Meta-Llama-3.1-8B-Instruct --include "original/*" --local-dir Meta-Llama-3.1-8B-Instruct
345
+ ```
346
 
347
  ## Hardware and Software
348