Initial README
Browse files# ezrkllm-collection
Collection of LLMs compatible with Rockchip's chips using their rkllm-toolkit. This repo contains the converted models for running on the RK3588 NPU found in SBCs like Orange Pi 5, NanoPi R6 and Radxa Rock 5.
## Available LLMs
Right now, only converted the following models:
- Qwen Chat (1.8B)
- Microsoft Phi-2 (2.7B)
However, RKLLM also supports Qwen 2 and Llama 2 7B, but I can't convert them due to my PC only having 16 GBs of RAM. For reference, converting Phi-2 peaked at about 15 GBs of RAM + 25 GBs of swap (counting OS, but it was using about 2 GBs max)
## Future additions
- [ ] Converting Qwen 2 and Llama 2
- [ ] Adding other compatible Rockchip's SoCs
## More info
My fork for rknn-llm: https://github.com/Pelochus/ezrknn-llm
Original Rockchip's LLMs repo: https://github.com/airockchip/rknn-llm