Commit
•
6ba3f19
1
Parent(s):
4f79769
Update README.md
Browse files
README.md
CHANGED
|
@@ -17,6 +17,20 @@ tags:
|
|
| 17 |
# DeepSeek-V2-Lite-Chat-IMat-GGUF
|
| 18 |
_Llama.cpp imatrix quantization of deepseek-ai/DeepSeek-V2-Lite-Chat_
|
| 19 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 20 |
Original Model: [deepseek-ai/DeepSeek-V2-Lite-Chat](https://huggingface.co/deepseek-ai/DeepSeek-V2-Lite-Chat)
|
| 21 |
Original dtype: `BF16` (`bfloat16`)
|
| 22 |
Quantized by: llama.cpp fork [PR 7519](https://github.com/ggerganov/llama.cpp/pull/7519)
|
|
|
|
| 17 |
# DeepSeek-V2-Lite-Chat-IMat-GGUF
|
| 18 |
_Llama.cpp imatrix quantization of deepseek-ai/DeepSeek-V2-Lite-Chat_
|
| 19 |
|
| 20 |
+
- [DeepSeek-V2-Lite-Chat-IMat-GGUF](#deepseek-v2-lite-chat-imat-gguf)
|
| 21 |
+
- [Files](#files)
|
| 22 |
+
- [IMatrix](#imatrix)
|
| 23 |
+
- [Common Quants](#common-quants)
|
| 24 |
+
- [All Quants](#all-quants)
|
| 25 |
+
- [Downloading using huggingface-cli](#downloading-using-huggingface-cli)
|
| 26 |
+
- [Inference](#inference)
|
| 27 |
+
- [Simple chat template](#simple-chat-template)
|
| 28 |
+
- [Chat template with system prompt](#chat-template-with-system-prompt)
|
| 29 |
+
- [Llama.cpp](#llama-cpp)
|
| 30 |
+
- [FAQ](#faq)
|
| 31 |
+
- [Why is the IMatrix not applied everywhere?](#why-is-the-imatrix-not-applied-everywhere)
|
| 32 |
+
- [How do I merge a split GGUF?](#how-do-i-merge-a-split-gguf)
|
| 33 |
+
|
| 34 |
Original Model: [deepseek-ai/DeepSeek-V2-Lite-Chat](https://huggingface.co/deepseek-ai/DeepSeek-V2-Lite-Chat)
|
| 35 |
Original dtype: `BF16` (`bfloat16`)
|
| 36 |
Quantized by: llama.cpp fork [PR 7519](https://github.com/ggerganov/llama.cpp/pull/7519)
|