Fixed Initial README
Browse filesI wrote the README in the extended description...
README.md
CHANGED
|
@@ -6,4 +6,22 @@ tags:
|
|
| 6 |
- rkllm
|
| 7 |
- phi2
|
| 8 |
- qwen
|
| 9 |
-
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 6 |
- rkllm
|
| 7 |
- phi2
|
| 8 |
- qwen
|
| 9 |
+
---
|
| 10 |
+
|
| 11 |
+
# ezrkllm-collection
|
| 12 |
+
Collection of LLMs compatible with Rockchip's chips using their rkllm-toolkit. This repo contains the converted models for running on the RK3588 NPU found in SBCs like Orange Pi 5, NanoPi R6 and Radxa Rock 5.
|
| 13 |
+
|
| 14 |
+
## Available LLMs
|
| 15 |
+
Right now, only converted the following models:
|
| 16 |
+
- Qwen Chat (1.8B)
|
| 17 |
+
- Microsoft Phi-2 (2.7B)
|
| 18 |
+
|
| 19 |
+
However, RKLLM also supports Qwen 2 and Llama 2 7B, but I can't convert them due to my PC only having 16 GBs of RAM. For reference, converting Phi-2 peaked at about 15 GBs of RAM + 25 GBs of swap (counting OS, but it was using about 2 GBs max)
|
| 20 |
+
|
| 21 |
+
## Future additions
|
| 22 |
+
- [ ] Converting Qwen 2 and Llama 2
|
| 23 |
+
- [ ] Adding other compatible Rockchip's SoCs
|
| 24 |
+
|
| 25 |
+
## More info
|
| 26 |
+
My fork for rknn-llm: https://github.com/Pelochus/ezrknn-llm
|
| 27 |
+
Original Rockchip's LLMs repo: https://github.com/airockchip/rknn-llm
|