Text Generation
GGUF
TensorBlock
GGUF
Eval Results
Inference Endpoints
morriszms commited on
Commit
5d97d66
1 Parent(s): d83cc0f

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +20 -12
README.md CHANGED
@@ -677,8 +677,16 @@ This repo contains GGUF format model files for [bigscience/bloomz-3b](https://hu
677
 
678
  The files were quantized using machines provided by [TensorBlock](https://tensorblock.co/), and they are compatible with llama.cpp as of [commit b4011](https://github.com/ggerganov/llama.cpp/commit/a6744e43e80f4be6398fc7733a01642c846dce1d).
679
 
 
 
 
 
 
 
 
680
  ## Prompt template
681
 
 
682
  ```
683
 
684
  ```
@@ -687,18 +695,18 @@ The files were quantized using machines provided by [TensorBlock](https://tensor
687
 
688
  | Filename | Quant type | File Size | Description |
689
  | -------- | ---------- | --------- | ----------- |
690
- | [bloomz-3b-Q2_K.gguf](https://huggingface.co/tensorblock/bloomz-3b-GGUF/tree/main/bloomz-3b-Q2_K.gguf) | Q2_K | 1.516 GB | smallest, significant quality loss - not recommended for most purposes |
691
- | [bloomz-3b-Q3_K_S.gguf](https://huggingface.co/tensorblock/bloomz-3b-GGUF/tree/main/bloomz-3b-Q3_K_S.gguf) | Q3_K_S | 1.707 GB | very small, high quality loss |
692
- | [bloomz-3b-Q3_K_M.gguf](https://huggingface.co/tensorblock/bloomz-3b-GGUF/tree/main/bloomz-3b-Q3_K_M.gguf) | Q3_K_M | 1.905 GB | very small, high quality loss |
693
- | [bloomz-3b-Q3_K_L.gguf](https://huggingface.co/tensorblock/bloomz-3b-GGUF/tree/main/bloomz-3b-Q3_K_L.gguf) | Q3_K_L | 2.016 GB | small, substantial quality loss |
694
- | [bloomz-3b-Q4_0.gguf](https://huggingface.co/tensorblock/bloomz-3b-GGUF/tree/main/bloomz-3b-Q4_0.gguf) | Q4_0 | 2.079 GB | legacy; small, very high quality loss - prefer using Q3_K_M |
695
- | [bloomz-3b-Q4_K_S.gguf](https://huggingface.co/tensorblock/bloomz-3b-GGUF/tree/main/bloomz-3b-Q4_K_S.gguf) | Q4_K_S | 2.088 GB | small, greater quality loss |
696
- | [bloomz-3b-Q4_K_M.gguf](https://huggingface.co/tensorblock/bloomz-3b-GGUF/tree/main/bloomz-3b-Q4_K_M.gguf) | Q4_K_M | 2.235 GB | medium, balanced quality - recommended |
697
- | [bloomz-3b-Q5_0.gguf](https://huggingface.co/tensorblock/bloomz-3b-GGUF/tree/main/bloomz-3b-Q5_0.gguf) | Q5_0 | 2.428 GB | legacy; medium, balanced quality - prefer using Q4_K_M |
698
- | [bloomz-3b-Q5_K_S.gguf](https://huggingface.co/tensorblock/bloomz-3b-GGUF/tree/main/bloomz-3b-Q5_K_S.gguf) | Q5_K_S | 2.428 GB | large, low quality loss - recommended |
699
- | [bloomz-3b-Q5_K_M.gguf](https://huggingface.co/tensorblock/bloomz-3b-GGUF/tree/main/bloomz-3b-Q5_K_M.gguf) | Q5_K_M | 2.546 GB | large, very low quality loss - recommended |
700
- | [bloomz-3b-Q6_K.gguf](https://huggingface.co/tensorblock/bloomz-3b-GGUF/tree/main/bloomz-3b-Q6_K.gguf) | Q6_K | 2.799 GB | very large, extremely low quality loss |
701
- | [bloomz-3b-Q8_0.gguf](https://huggingface.co/tensorblock/bloomz-3b-GGUF/tree/main/bloomz-3b-Q8_0.gguf) | Q8_0 | 3.621 GB | very large, extremely low quality loss - not recommended |
702
 
703
 
704
  ## Downloading instruction
 
677
 
678
  The files were quantized using machines provided by [TensorBlock](https://tensorblock.co/), and they are compatible with llama.cpp as of [commit b4011](https://github.com/ggerganov/llama.cpp/commit/a6744e43e80f4be6398fc7733a01642c846dce1d).
679
 
680
+
681
+ <div style="text-align: left; margin: 20px 0;">
682
+ <a href="https://tensorblock.co/waitlist/client" style="display: inline-block; padding: 10px 20px; background-color: #007bff; color: white; text-decoration: none; border-radius: 5px; font-weight: bold;">
683
+ Run them on the TensorBlock client using your local machine ↗
684
+ </a>
685
+ </div>
686
+
687
  ## Prompt template
688
 
689
+
690
  ```
691
 
692
  ```
 
695
 
696
  | Filename | Quant type | File Size | Description |
697
  | -------- | ---------- | --------- | ----------- |
698
+ | [bloomz-3b-Q2_K.gguf](https://huggingface.co/tensorblock/bloomz-3b-GGUF/blob/main/bloomz-3b-Q2_K.gguf) | Q2_K | 1.516 GB | smallest, significant quality loss - not recommended for most purposes |
699
+ | [bloomz-3b-Q3_K_S.gguf](https://huggingface.co/tensorblock/bloomz-3b-GGUF/blob/main/bloomz-3b-Q3_K_S.gguf) | Q3_K_S | 1.707 GB | very small, high quality loss |
700
+ | [bloomz-3b-Q3_K_M.gguf](https://huggingface.co/tensorblock/bloomz-3b-GGUF/blob/main/bloomz-3b-Q3_K_M.gguf) | Q3_K_M | 1.905 GB | very small, high quality loss |
701
+ | [bloomz-3b-Q3_K_L.gguf](https://huggingface.co/tensorblock/bloomz-3b-GGUF/blob/main/bloomz-3b-Q3_K_L.gguf) | Q3_K_L | 2.016 GB | small, substantial quality loss |
702
+ | [bloomz-3b-Q4_0.gguf](https://huggingface.co/tensorblock/bloomz-3b-GGUF/blob/main/bloomz-3b-Q4_0.gguf) | Q4_0 | 2.079 GB | legacy; small, very high quality loss - prefer using Q3_K_M |
703
+ | [bloomz-3b-Q4_K_S.gguf](https://huggingface.co/tensorblock/bloomz-3b-GGUF/blob/main/bloomz-3b-Q4_K_S.gguf) | Q4_K_S | 2.088 GB | small, greater quality loss |
704
+ | [bloomz-3b-Q4_K_M.gguf](https://huggingface.co/tensorblock/bloomz-3b-GGUF/blob/main/bloomz-3b-Q4_K_M.gguf) | Q4_K_M | 2.235 GB | medium, balanced quality - recommended |
705
+ | [bloomz-3b-Q5_0.gguf](https://huggingface.co/tensorblock/bloomz-3b-GGUF/blob/main/bloomz-3b-Q5_0.gguf) | Q5_0 | 2.428 GB | legacy; medium, balanced quality - prefer using Q4_K_M |
706
+ | [bloomz-3b-Q5_K_S.gguf](https://huggingface.co/tensorblock/bloomz-3b-GGUF/blob/main/bloomz-3b-Q5_K_S.gguf) | Q5_K_S | 2.428 GB | large, low quality loss - recommended |
707
+ | [bloomz-3b-Q5_K_M.gguf](https://huggingface.co/tensorblock/bloomz-3b-GGUF/blob/main/bloomz-3b-Q5_K_M.gguf) | Q5_K_M | 2.546 GB | large, very low quality loss - recommended |
708
+ | [bloomz-3b-Q6_K.gguf](https://huggingface.co/tensorblock/bloomz-3b-GGUF/blob/main/bloomz-3b-Q6_K.gguf) | Q6_K | 2.799 GB | very large, extremely low quality loss |
709
+ | [bloomz-3b-Q8_0.gguf](https://huggingface.co/tensorblock/bloomz-3b-GGUF/blob/main/bloomz-3b-Q8_0.gguf) | Q8_0 | 3.621 GB | very large, extremely low quality loss - not recommended |
710
 
711
 
712
  ## Downloading instruction