PY007 commited on
Commit
555570e
•
1 Parent(s): 849bf2b

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -28
README.md CHANGED
@@ -28,31 +28,4 @@ This is a code LM finetuned(or so-called continue pretrianed) from the 500B Tiny
28
 
29
  The HumanEval accuracy is **14**.
30
 
31
- #### How to use
32
- You will need the transformers>=4.31
33
- Do check the [TinyLlama](https://github.com/jzhang38/TinyLlama) github page for more information.
34
- ```
35
- from transformers import AutoTokenizer
36
- import transformers
37
- import torch
38
- model = "PY007/TinyLlama-1.1B-intermediate-step-240k-503b"
39
- tokenizer = AutoTokenizer.from_pretrained(model)
40
- pipeline = transformers.pipeline(
41
- "text-generation",
42
- model=model,
43
- torch_dtype=torch.float16,
44
- device_map="auto",
45
- )
46
-
47
- sequences = pipeline(
48
- 'The TinyLlama project aims to pretrain a 1.1B Llama model on 3 trillion tokens. With some proper optimization, we can achieve this within a span of "just" 90 days using 16 A100-40G GPUs 🚀🚀. The training has started on 2023-09-01.',
49
- do_sample=True,
50
- top_k=10,
51
- num_return_sequences=1,
52
- repetition_penalty=1.5,
53
- eos_token_id=tokenizer.eos_token_id,
54
- max_length=500,
55
- )
56
- for seq in sequences:
57
- print(f"Result: {seq['generated_text']}")
58
- ```
 
28
 
29
  The HumanEval accuracy is **14**.
30
 
31
+ **It can be used as the draft model to speculative-decode larger models such as models in the CodeLlama family**.