Fix the model identifier
#2
by
Felladrin
- opened
README.md
CHANGED
@@ -10,7 +10,7 @@ License: Must comply with license of Pythia since it's a model derived from Pyth
|
|
10 |
Sheared-Pythia-160m is a model pruned and further pre-trained from [EleutherAI/pythia-410m](https://huggingface.co/EleutherAI/pythia-410m). We dynamically load data from different domains in the Pile dataset to prune and contune pre-train the model. We use 0.4B tokens for pruning and 50B tokens for continued pre-training the pruned model. This model can be loaded with HuggingFace via
|
11 |
|
12 |
```
|
13 |
-
model = GPTNeoXForCausalLM.from_pretrained("princeton-nlp/Sheared-Pythia-
|
14 |
```
|
15 |
|
16 |
The model's overall performance is better than EleutherAI/pythia-160m.
|
|
|
10 |
Sheared-Pythia-160m is a model pruned and further pre-trained from [EleutherAI/pythia-410m](https://huggingface.co/EleutherAI/pythia-410m). We dynamically load data from different domains in the Pile dataset to prune and contune pre-train the model. We use 0.4B tokens for pruning and 50B tokens for continued pre-training the pruned model. This model can be loaded with HuggingFace via
|
11 |
|
12 |
```
|
13 |
+
model = GPTNeoXForCausalLM.from_pretrained("princeton-nlp/Sheared-Pythia-160m")
|
14 |
```
|
15 |
|
16 |
The model's overall performance is better than EleutherAI/pythia-160m.
|