Update README.md
Browse files
README.md
CHANGED
@@ -17,7 +17,12 @@ They're separated by date and commit so it's easier to track any breaking change
|
|
17 |
|
18 |
If you're here because you want a smaller model to run on a device with constrained memory, try the instruct-based RWKV-Raven ([q8_0](https://huggingface.co/BlinkDL/rwkv-4-raven/tree/main) and [q5_1](https://huggingface.co/latestissue/RWKV-4-Raven-CPP-Converted-Quantized/tree/main)) which goes as low as 1.5B, or [RWKV-PilePlus](https://huggingface.co/BlinkDL/rwkv-4-pileplus/tree/main), which goes as low as 169M.
|
19 |
|
20 |
-
If you're here because you want an openly-licensed LLaMA,
|
|
|
|
|
|
|
|
|
|
|
21 |
|
22 |
# RAM USAGE
|
23 |
Model | Initial RAM usage
|
|
|
17 |
|
18 |
If you're here because you want a smaller model to run on a device with constrained memory, try the instruct-based RWKV-Raven ([q8_0](https://huggingface.co/BlinkDL/rwkv-4-raven/tree/main) and [q5_1](https://huggingface.co/latestissue/RWKV-4-Raven-CPP-Converted-Quantized/tree/main)) which goes as low as 1.5B, or [RWKV-PilePlus](https://huggingface.co/BlinkDL/rwkv-4-pileplus/tree/main), which goes as low as 169M.
|
19 |
|
20 |
+
If you're here because you want an openly-licensed LLaMA, there's:
|
21 |
+
- OpenLLaMA [(7B)](https://huggingface.co/openlm-research/open_llama_7b_preview_300bt)
|
22 |
+
- RedPajama-INCITE [(3B)](https://huggingface.co/togethercomputer/RedPajama-INCITE-Base-3B-v1) [(7B)](https://huggingface.co/togethercomputer/RedPajama-INCITE-Base-7B-v0.1)
|
23 |
+
- MPT [(1B)](https://huggingface.co/mosaicml/mpt-1b-redpajama-200b) [(7B)](https://huggingface.co/mosaicml/mpt-7b).
|
24 |
+
|
25 |
+
All of them are trained on an open reproduction of LLaMA's dataset, [RedPajama](https://huggingface.co/datasets/togethercomputer/RedPajama-Data-1T), but they're based on different architectures. OpenLLaMA is based on the LLaMA architecture (making it compatible with llama.cpp), RedPajama-INCITE is based on GPT-NeoX, and MPT uses its own.
|
26 |
|
27 |
# RAM USAGE
|
28 |
Model | Initial RAM usage
|