Hugging Face's logo

c01zaut
/
Qwen2.5-7B-Instruct-RK3588-1.1.4

Text Generation
Transformers
Safetensors
English
qwen2
chat
conversational
text-generation-inference
Inference Endpoints
Model card Files Community
c01zaut commited on
Commit
592f25d
•
1 Parent(s): 99fdf81

Upload folder using huggingface_hub

Browse files
Files changed (26) hide show
  1. .gitattributes +18 -0
  2. Qwen2.5-7B-Instruct-rk3588-w8a8-opt-0-hybrid-ratio-0.0.rkllm +1 -1
  3. Qwen2.5-7B-Instruct-rk3588-w8a8-opt-0-hybrid-ratio-0.5.rkllm +1 -1
  4. Qwen2.5-7B-Instruct-rk3588-w8a8-opt-0-hybrid-ratio-1.0.rkllm +1 -1
  5. Qwen2.5-7B-Instruct-rk3588-w8a8-opt-1-hybrid-ratio-0.0.rkllm +1 -1
  6. Qwen2.5-7B-Instruct-rk3588-w8a8-opt-1-hybrid-ratio-0.5.rkllm +1 -1
  7. Qwen2.5-7B-Instruct-rk3588-w8a8-opt-1-hybrid-ratio-1.0.rkllm +1 -1
  8. Qwen2.5-7B-Instruct-rk3588-w8a8_g128-opt-0-hybrid-ratio-0.0.rkllm +3 -0
  9. Qwen2.5-7B-Instruct-rk3588-w8a8_g128-opt-0-hybrid-ratio-0.5.rkllm +3 -0
  10. Qwen2.5-7B-Instruct-rk3588-w8a8_g128-opt-0-hybrid-ratio-1.0.rkllm +3 -0
  11. Qwen2.5-7B-Instruct-rk3588-w8a8_g128-opt-1-hybrid-ratio-0.0.rkllm +3 -0
  12. Qwen2.5-7B-Instruct-rk3588-w8a8_g128-opt-1-hybrid-ratio-0.5.rkllm +3 -0
  13. Qwen2.5-7B-Instruct-rk3588-w8a8_g128-opt-1-hybrid-ratio-1.0.rkllm +3 -0
  14. Qwen2.5-7B-Instruct-rk3588-w8a8_g256-opt-0-hybrid-ratio-0.0.rkllm +3 -0
  15. Qwen2.5-7B-Instruct-rk3588-w8a8_g256-opt-0-hybrid-ratio-0.5.rkllm +3 -0
  16. Qwen2.5-7B-Instruct-rk3588-w8a8_g256-opt-0-hybrid-ratio-1.0.rkllm +3 -0
  17. Qwen2.5-7B-Instruct-rk3588-w8a8_g256-opt-1-hybrid-ratio-0.0.rkllm +3 -0
  18. Qwen2.5-7B-Instruct-rk3588-w8a8_g256-opt-1-hybrid-ratio-0.5.rkllm +3 -0
  19. Qwen2.5-7B-Instruct-rk3588-w8a8_g256-opt-1-hybrid-ratio-1.0.rkllm +3 -0
  20. Qwen2.5-7B-Instruct-rk3588-w8a8_g512-opt-0-hybrid-ratio-0.0.rkllm +3 -0
  21. Qwen2.5-7B-Instruct-rk3588-w8a8_g512-opt-0-hybrid-ratio-0.5.rkllm +3 -0
  22. Qwen2.5-7B-Instruct-rk3588-w8a8_g512-opt-0-hybrid-ratio-1.0.rkllm +3 -0
  23. Qwen2.5-7B-Instruct-rk3588-w8a8_g512-opt-1-hybrid-ratio-0.0.rkllm +3 -0
  24. Qwen2.5-7B-Instruct-rk3588-w8a8_g512-opt-1-hybrid-ratio-0.5.rkllm +3 -0
  25. Qwen2.5-7B-Instruct-rk3588-w8a8_g512-opt-1-hybrid-ratio-1.0.rkllm +3 -0
  26. README.md +1 -1
.gitattributes CHANGED
@@ -39,3 +39,21 @@ Qwen2.5-7B-Instruct-rk3588-w8a8-opt-0-hybrid-ratio-1.0.rkllm filter=lfs diff=lfs
39
  Qwen2.5-7B-Instruct-rk3588-w8a8-opt-1-hybrid-ratio-0.0.rkllm filter=lfs diff=lfs merge=lfs -text
40
  Qwen2.5-7B-Instruct-rk3588-w8a8-opt-1-hybrid-ratio-0.5.rkllm filter=lfs diff=lfs merge=lfs -text
41
  Qwen2.5-7B-Instruct-rk3588-w8a8-opt-1-hybrid-ratio-1.0.rkllm filter=lfs diff=lfs merge=lfs -text
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
39
  Qwen2.5-7B-Instruct-rk3588-w8a8-opt-1-hybrid-ratio-0.0.rkllm filter=lfs diff=lfs merge=lfs -text
40
  Qwen2.5-7B-Instruct-rk3588-w8a8-opt-1-hybrid-ratio-0.5.rkllm filter=lfs diff=lfs merge=lfs -text
41
  Qwen2.5-7B-Instruct-rk3588-w8a8-opt-1-hybrid-ratio-1.0.rkllm filter=lfs diff=lfs merge=lfs -text
42
+ Qwen2.5-7B-Instruct-rk3588-w8a8_g128-opt-0-hybrid-ratio-0.0.rkllm filter=lfs diff=lfs merge=lfs -text
43
+ Qwen2.5-7B-Instruct-rk3588-w8a8_g128-opt-0-hybrid-ratio-0.5.rkllm filter=lfs diff=lfs merge=lfs -text
44
+ Qwen2.5-7B-Instruct-rk3588-w8a8_g128-opt-0-hybrid-ratio-1.0.rkllm filter=lfs diff=lfs merge=lfs -text
45
+ Qwen2.5-7B-Instruct-rk3588-w8a8_g128-opt-1-hybrid-ratio-0.0.rkllm filter=lfs diff=lfs merge=lfs -text
46
+ Qwen2.5-7B-Instruct-rk3588-w8a8_g128-opt-1-hybrid-ratio-0.5.rkllm filter=lfs diff=lfs merge=lfs -text
47
+ Qwen2.5-7B-Instruct-rk3588-w8a8_g128-opt-1-hybrid-ratio-1.0.rkllm filter=lfs diff=lfs merge=lfs -text
48
+ Qwen2.5-7B-Instruct-rk3588-w8a8_g256-opt-0-hybrid-ratio-0.0.rkllm filter=lfs diff=lfs merge=lfs -text
49
+ Qwen2.5-7B-Instruct-rk3588-w8a8_g256-opt-0-hybrid-ratio-0.5.rkllm filter=lfs diff=lfs merge=lfs -text
50
+ Qwen2.5-7B-Instruct-rk3588-w8a8_g256-opt-0-hybrid-ratio-1.0.rkllm filter=lfs diff=lfs merge=lfs -text
51
+ Qwen2.5-7B-Instruct-rk3588-w8a8_g256-opt-1-hybrid-ratio-0.0.rkllm filter=lfs diff=lfs merge=lfs -text
52
+ Qwen2.5-7B-Instruct-rk3588-w8a8_g256-opt-1-hybrid-ratio-0.5.rkllm filter=lfs diff=lfs merge=lfs -text
53
+ Qwen2.5-7B-Instruct-rk3588-w8a8_g256-opt-1-hybrid-ratio-1.0.rkllm filter=lfs diff=lfs merge=lfs -text
54
+ Qwen2.5-7B-Instruct-rk3588-w8a8_g512-opt-0-hybrid-ratio-0.0.rkllm filter=lfs diff=lfs merge=lfs -text
55
+ Qwen2.5-7B-Instruct-rk3588-w8a8_g512-opt-0-hybrid-ratio-0.5.rkllm filter=lfs diff=lfs merge=lfs -text
56
+ Qwen2.5-7B-Instruct-rk3588-w8a8_g512-opt-0-hybrid-ratio-1.0.rkllm filter=lfs diff=lfs merge=lfs -text
57
+ Qwen2.5-7B-Instruct-rk3588-w8a8_g512-opt-1-hybrid-ratio-0.0.rkllm filter=lfs diff=lfs merge=lfs -text
58
+ Qwen2.5-7B-Instruct-rk3588-w8a8_g512-opt-1-hybrid-ratio-0.5.rkllm filter=lfs diff=lfs merge=lfs -text
59
+ Qwen2.5-7B-Instruct-rk3588-w8a8_g512-opt-1-hybrid-ratio-1.0.rkllm filter=lfs diff=lfs merge=lfs -text
Qwen2.5-7B-Instruct-rk3588-w8a8-opt-0-hybrid-ratio-0.0.rkllm CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:def6ea44d396705814b37d281d3d1085306503fc66989d987d749c8c4fb69f0e
3
  size 8193646756
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7a56ff851df2e0716f55e9b6bc2310ea6ed0478bb624e75a497fbf3b65bc89b9
3
  size 8193646756
Qwen2.5-7B-Instruct-rk3588-w8a8-opt-0-hybrid-ratio-0.5.rkllm CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:def6ea44d396705814b37d281d3d1085306503fc66989d987d749c8c4fb69f0e
3
  size 8193646756
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7a56ff851df2e0716f55e9b6bc2310ea6ed0478bb624e75a497fbf3b65bc89b9
3
  size 8193646756
Qwen2.5-7B-Instruct-rk3588-w8a8-opt-0-hybrid-ratio-1.0.rkllm CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:def6ea44d396705814b37d281d3d1085306503fc66989d987d749c8c4fb69f0e
3
  size 8193646756
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7a56ff851df2e0716f55e9b6bc2310ea6ed0478bb624e75a497fbf3b65bc89b9
3
  size 8193646756
Qwen2.5-7B-Instruct-rk3588-w8a8-opt-1-hybrid-ratio-0.0.rkllm CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:a0c74f3effc54b3a59b9a82748079f2740713bbfe5092f874c3e5ae4f2419266
3
  size 8193646756
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:820210d78d46acdd0c56ba49045a86db38d9f4d542253a24bf64110ae1888a75
3
  size 8193646756
Qwen2.5-7B-Instruct-rk3588-w8a8-opt-1-hybrid-ratio-0.5.rkllm CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:a0c74f3effc54b3a59b9a82748079f2740713bbfe5092f874c3e5ae4f2419266
3
  size 8193646756
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:820210d78d46acdd0c56ba49045a86db38d9f4d542253a24bf64110ae1888a75
3
  size 8193646756
Qwen2.5-7B-Instruct-rk3588-w8a8-opt-1-hybrid-ratio-1.0.rkllm CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:a0c74f3effc54b3a59b9a82748079f2740713bbfe5092f874c3e5ae4f2419266
3
  size 8193646756
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:820210d78d46acdd0c56ba49045a86db38d9f4d542253a24bf64110ae1888a75
3
  size 8193646756
Qwen2.5-7B-Instruct-rk3588-w8a8_g128-opt-0-hybrid-ratio-0.0.rkllm ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3a2469b3c5e6aff275f049fc56255160c00e5f61781a334e0bb1ef8b679bbad0
3
+ size 8685611588
Qwen2.5-7B-Instruct-rk3588-w8a8_g128-opt-0-hybrid-ratio-0.5.rkllm ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3a2469b3c5e6aff275f049fc56255160c00e5f61781a334e0bb1ef8b679bbad0
3
+ size 8685611588
Qwen2.5-7B-Instruct-rk3588-w8a8_g128-opt-0-hybrid-ratio-1.0.rkllm ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3a2469b3c5e6aff275f049fc56255160c00e5f61781a334e0bb1ef8b679bbad0
3
+ size 8685611588
Qwen2.5-7B-Instruct-rk3588-w8a8_g128-opt-1-hybrid-ratio-0.0.rkllm ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:241dd6b0301a52dca27eeab434ee803be3d6073b48aa7c50a82759eb3d04f7d8
3
+ size 8651216572
Qwen2.5-7B-Instruct-rk3588-w8a8_g128-opt-1-hybrid-ratio-0.5.rkllm ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:241dd6b0301a52dca27eeab434ee803be3d6073b48aa7c50a82759eb3d04f7d8
3
+ size 8651216572
Qwen2.5-7B-Instruct-rk3588-w8a8_g128-opt-1-hybrid-ratio-1.0.rkllm ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:241dd6b0301a52dca27eeab434ee803be3d6073b48aa7c50a82759eb3d04f7d8
3
+ size 8651216572
Qwen2.5-7B-Instruct-rk3588-w8a8_g256-opt-0-hybrid-ratio-0.0.rkllm ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:886a4f1831bf622c2d41968682cbd357d5bafd264ae5fae8d6e2d60afd6d59d8
3
+ size 8433071044
Qwen2.5-7B-Instruct-rk3588-w8a8_g256-opt-0-hybrid-ratio-0.5.rkllm ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:886a4f1831bf622c2d41968682cbd357d5bafd264ae5fae8d6e2d60afd6d59d8
3
+ size 8433071044
Qwen2.5-7B-Instruct-rk3588-w8a8_g256-opt-0-hybrid-ratio-1.0.rkllm ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:886a4f1831bf622c2d41968682cbd357d5bafd264ae5fae8d6e2d60afd6d59d8
3
+ size 8433071044
Qwen2.5-7B-Instruct-rk3588-w8a8_g256-opt-1-hybrid-ratio-0.0.rkllm ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:626bd0936c15cd5507a0bb14301ec293217f7fea319e3c03c3342ae8042283d9
3
+ size 8416510244
Qwen2.5-7B-Instruct-rk3588-w8a8_g256-opt-1-hybrid-ratio-0.5.rkllm ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:626bd0936c15cd5507a0bb14301ec293217f7fea319e3c03c3342ae8042283d9
3
+ size 8416510244
Qwen2.5-7B-Instruct-rk3588-w8a8_g256-opt-1-hybrid-ratio-1.0.rkllm ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:626bd0936c15cd5507a0bb14301ec293217f7fea319e3c03c3342ae8042283d9
3
+ size 8416510244
Qwen2.5-7B-Instruct-rk3588-w8a8_g512-opt-0-hybrid-ratio-0.0.rkllm ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8a5c64d2497b4bc001f335c95f7ebc456559b234003a6b23bfcd80998a8b5694
3
+ size 8306800748
Qwen2.5-7B-Instruct-rk3588-w8a8_g512-opt-0-hybrid-ratio-0.5.rkllm ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8a5c64d2497b4bc001f335c95f7ebc456559b234003a6b23bfcd80998a8b5694
3
+ size 8306800748
Qwen2.5-7B-Instruct-rk3588-w8a8_g512-opt-0-hybrid-ratio-1.0.rkllm ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8a5c64d2497b4bc001f335c95f7ebc456559b234003a6b23bfcd80998a8b5694
3
+ size 8306800748
Qwen2.5-7B-Instruct-rk3588-w8a8_g512-opt-1-hybrid-ratio-0.0.rkllm ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ce11d1bbe946b0ffbb12b862c84ad3d18e441c1fb70088dbc4ac6fcf8d39c4d7
3
+ size 8299157044
Qwen2.5-7B-Instruct-rk3588-w8a8_g512-opt-1-hybrid-ratio-0.5.rkllm ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ce11d1bbe946b0ffbb12b862c84ad3d18e441c1fb70088dbc4ac6fcf8d39c4d7
3
+ size 8299157044
Qwen2.5-7B-Instruct-rk3588-w8a8_g512-opt-1-hybrid-ratio-1.0.rkllm ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ce11d1bbe946b0ffbb12b862c84ad3d18e441c1fb70088dbc4ac6fcf8d39c4d7
3
+ size 8299157044
README.md CHANGED
@@ -11,7 +11,7 @@ tags:
11
  ---
12
  # Qwen2.5-7B-Instruct-RK3588-1.1.2
13
 
14
- This version of Qwen2.5-7B-Instruct has been converted to run on the RK3588 NPU using ['w8a8'] quantization.
15
  This model has been optimized with the following LoRA:
16
 
17
  Compatible with RKLLM version: 1.1.2
 
11
  ---
12
  # Qwen2.5-7B-Instruct-RK3588-1.1.2
13
 
14
+ This version of Qwen2.5-7B-Instruct has been converted to run on the RK3588 NPU using ['w8a8', 'w8a8_g128', 'w8a8_g256', 'w8a8_g512'] quantization.
15
  This model has been optimized with the following LoRA:
16
 
17
  Compatible with RKLLM version: 1.1.2