roleplaiapp
commited on
Update README.md
Browse files
README.md
CHANGED
@@ -12,7 +12,7 @@ tags:
|
|
12 |
- llama-cpp
|
13 |
- imatrix
|
14 |
- gguf
|
15 |
-
-
|
16 |
- 3b
|
17 |
- SmallThinker
|
18 |
- qwen
|
@@ -31,16 +31,16 @@ tags:
|
|
31 |
# roleplaiapp/SmallThinker-3B-Preview-IQ4_NL-GGUF
|
32 |
|
33 |
**Repo:** `roleplaiapp/SmallThinker-3B-Preview-IQ4_NL-GGUF`
|
34 |
-
**Original Model:** `
|
35 |
**Organization:** `PowerInfer`
|
36 |
-
**Quantized File:** `smallthinker-3b-preview-
|
37 |
**Quantization:** `GGUF`
|
38 |
-
**Quantization Method:** `
|
39 |
**Use Imatrix:** `True`
|
40 |
**Split Model:** `False`
|
41 |
|
42 |
## Overview
|
43 |
-
This is an GGUF
|
44 |
|
45 |
## Quantization By
|
46 |
I often have idle A100 GPUs while building/testing and training the RP app, so I put them to use quantizing models.
|
|
|
12 |
- llama-cpp
|
13 |
- imatrix
|
14 |
- gguf
|
15 |
+
- IQ4_NL
|
16 |
- 3b
|
17 |
- SmallThinker
|
18 |
- qwen
|
|
|
31 |
# roleplaiapp/SmallThinker-3B-Preview-IQ4_NL-GGUF
|
32 |
|
33 |
**Repo:** `roleplaiapp/SmallThinker-3B-Preview-IQ4_NL-GGUF`
|
34 |
+
**Original Model:** `SmallThinker-3B-Preview`
|
35 |
**Organization:** `PowerInfer`
|
36 |
+
**Quantized File:** `smallthinker-3b-preview-iq_4nl-imat.gguf`
|
37 |
**Quantization:** `GGUF`
|
38 |
+
**Quantization Method:** `IQ4_NL`
|
39 |
**Use Imatrix:** `True`
|
40 |
**Split Model:** `False`
|
41 |
|
42 |
## Overview
|
43 |
+
This is an imatrix GGUF IQ4_NL quantized version of [SmallThinker-3B-Preview](https://huggingface.co/PowerInfer/SmallThinker-3B-Preview).
|
44 |
|
45 |
## Quantization By
|
46 |
I often have idle A100 GPUs while building/testing and training the RP app, so I put them to use quantizing models.
|