Pelochus commited on
Commit
24121d0
1 Parent(s): b3b19e1

Better README

Browse files
Files changed (1) hide show
  1. README.md +9 -6
README.md CHANGED
@@ -4,21 +4,24 @@ tags:
4
  - rockchip
5
  - rk3588
6
  - rkllm
7
- - phi2
8
- - qwen
9
  - text-generation-inference
10
  pipeline_tag: text-generation
11
  ---
12
 
13
  # ezrkllm-collection
14
- Collection of LLMs compatible with Rockchip's chips using their rkllm-toolkit. This repo contains the converted models for running on the RK3588 NPU found in SBCs like Orange Pi 5, NanoPi R6 and Radxa Rock 5.
 
15
 
16
  ## Available LLMs
17
  Right now, only converted the following models:
18
- - Qwen Chat (1.8B)
19
- - Microsoft Phi-2 (2.7B)
 
 
 
20
 
21
- However, RKLLM also supports Qwen 2 and Llama 2 7B, but I can't convert them due to my PC only having 16 GBs of RAM. For reference, converting Phi-2 peaked at about 15 GBs of RAM + 25 GBs of swap (counting OS, but it was using about 2 GBs max)
 
22
 
23
  ## Future additions
24
  - [ ] Converting Qwen 2 and Llama 2
 
4
  - rockchip
5
  - rk3588
6
  - rkllm
 
 
7
  - text-generation-inference
8
  pipeline_tag: text-generation
9
  ---
10
 
11
  # ezrkllm-collection
12
+ Collection of LLMs compatible with Rockchip's chips using their rkllm-toolkit.
13
+ This repo contains the converted models for running on the RK3588 NPU found in SBCs like Orange Pi 5, NanoPi R6 and Radxa Rock 5.
14
 
15
  ## Available LLMs
16
  Right now, only converted the following models:
17
+ | LLM | Parameters | Link |
18
+ | --------------------- | ----------- | ---------------------------------------------------------- |
19
+ | Qwen Chat | 1.8B | https://huggingface.co/Pelochus/qwen-1_8B-rk3588/tree/main |
20
+ | Microsoft Phi-2 | 2.7B | https://huggingface.co/Pelochus/phi-2-rk3588/tree/main |
21
+ | TinyLlama v1 | 1.1B | TODO |
22
 
23
+ However, RKLLM also supports Qwen 2 and Llama 2 7B, but I can't convert them due to my PC only having 16 GBs of RAM.
24
+ For reference, converting Phi-2 peaked at about 15 GBs of RAM + 25 GBs of swap (counting OS, but that was using about 2 GBs max).
25
 
26
  ## Future additions
27
  - [ ] Converting Qwen 2 and Llama 2