roleplaiapp commited on
Commit
1dad0d7
·
verified ·
1 Parent(s): 23f2b3d

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +5 -5
README.md CHANGED
@@ -12,7 +12,7 @@ tags:
12
  - llama-cpp
13
  - imatrix
14
  - gguf
15
- - Q8_0
16
  - 3b
17
  - SmallThinker
18
  - qwen
@@ -31,16 +31,16 @@ tags:
31
  # roleplaiapp/SmallThinker-3B-Preview-IQ4_NL-GGUF
32
 
33
  **Repo:** `roleplaiapp/SmallThinker-3B-Preview-IQ4_NL-GGUF`
34
- **Original Model:** `imatrix`
35
  **Organization:** `PowerInfer`
36
- **Quantized File:** `smallthinker-3b-preview-iq4_nl-imat.gguf`
37
  **Quantization:** `GGUF`
38
- **Quantization Method:** `Q8_0`
39
  **Use Imatrix:** `True`
40
  **Split Model:** `False`
41
 
42
  ## Overview
43
- This is an GGUF Q8_0 quantized version of [imatrix](https://huggingface.co/PowerInfer/SmallThinker-3B-Preview).
44
 
45
  ## Quantization By
46
  I often have idle A100 GPUs while building/testing and training the RP app, so I put them to use quantizing models.
 
12
  - llama-cpp
13
  - imatrix
14
  - gguf
15
+ - IQ4_NL
16
  - 3b
17
  - SmallThinker
18
  - qwen
 
31
  # roleplaiapp/SmallThinker-3B-Preview-IQ4_NL-GGUF
32
 
33
  **Repo:** `roleplaiapp/SmallThinker-3B-Preview-IQ4_NL-GGUF`
34
+ **Original Model:** `SmallThinker-3B-Preview`
35
  **Organization:** `PowerInfer`
36
+ **Quantized File:** `smallthinker-3b-preview-iq_4nl-imat.gguf`
37
  **Quantization:** `GGUF`
38
+ **Quantization Method:** `IQ4_NL`
39
  **Use Imatrix:** `True`
40
  **Split Model:** `False`
41
 
42
  ## Overview
43
+ This is an imatrix GGUF IQ4_NL quantized version of [SmallThinker-3B-Preview](https://huggingface.co/PowerInfer/SmallThinker-3B-Preview).
44
 
45
  ## Quantization By
46
  I often have idle A100 GPUs while building/testing and training the RP app, so I put them to use quantizing models.