Better README
Browse files
README.md
CHANGED
@@ -4,21 +4,24 @@ tags:
|
|
4 |
- rockchip
|
5 |
- rk3588
|
6 |
- rkllm
|
7 |
-
- phi2
|
8 |
-
- qwen
|
9 |
- text-generation-inference
|
10 |
pipeline_tag: text-generation
|
11 |
---
|
12 |
|
13 |
# ezrkllm-collection
|
14 |
-
Collection of LLMs compatible with Rockchip's chips using their rkllm-toolkit.
|
|
|
15 |
|
16 |
## Available LLMs
|
17 |
Right now, only converted the following models:
|
18 |
-
|
19 |
-
|
|
|
|
|
|
|
20 |
|
21 |
-
However, RKLLM also supports Qwen 2 and Llama 2 7B, but I can't convert them due to my PC only having 16 GBs of RAM.
|
|
|
22 |
|
23 |
## Future additions
|
24 |
- [ ] Converting Qwen 2 and Llama 2
|
|
|
4 |
- rockchip
|
5 |
- rk3588
|
6 |
- rkllm
|
|
|
|
|
7 |
- text-generation-inference
|
8 |
pipeline_tag: text-generation
|
9 |
---
|
10 |
|
11 |
# ezrkllm-collection
|
12 |
+
Collection of LLMs compatible with Rockchip's chips using their rkllm-toolkit.
|
13 |
+
This repo contains the converted models for running on the RK3588 NPU found in SBCs like Orange Pi 5, NanoPi R6 and Radxa Rock 5.
|
14 |
|
15 |
## Available LLMs
|
16 |
Right now, only converted the following models:
|
17 |
+
| LLM | Parameters | Link |
|
18 |
+
| --------------------- | ----------- | ---------------------------------------------------------- |
|
19 |
+
| Qwen Chat | 1.8B | https://huggingface.co/Pelochus/qwen-1_8B-rk3588/tree/main |
|
20 |
+
| Microsoft Phi-2 | 2.7B | https://huggingface.co/Pelochus/phi-2-rk3588/tree/main |
|
21 |
+
| TinyLlama v1 | 1.1B | TODO |
|
22 |
|
23 |
+
However, RKLLM also supports Qwen 2 and Llama 2 7B, but I can't convert them due to my PC only having 16 GBs of RAM.
|
24 |
+
For reference, converting Phi-2 peaked at about 15 GBs of RAM + 25 GBs of swap (counting OS, but that was using about 2 GBs max).
|
25 |
|
26 |
## Future additions
|
27 |
- [ ] Converting Qwen 2 and Llama 2
|