Update README.md
Browse files
README.md
CHANGED
|
@@ -46,6 +46,12 @@ Please note that these GGMLs are **not compatible with llama.cpp, or currently w
|
|
| 46 |
|
| 47 |
[KoboldCpp](https://github.com/LostRuins/koboldcpp) just added GPU accelerated (OpenCL) support for MPT models, so that is the client I recommend using for these models.
|
| 48 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 49 |
## Repositories available
|
| 50 |
|
| 51 |
* [2, 3, 4, 5, 6 and 8-bit GGML models for CPU+GPU inference](https://huggingface.co/TheBloke/mpt-30B-chat-GGML)
|
|
|
|
| 46 |
|
| 47 |
[KoboldCpp](https://github.com/LostRuins/koboldcpp) just added GPU accelerated (OpenCL) support for MPT models, so that is the client I recommend using for these models.
|
| 48 |
|
| 49 |
+
**Note**: There is currently a bug with loading this model in KoboldCpp Release 1.32: it will wrongly detect it as a GPT-NeoX model.
|
| 50 |
+
|
| 51 |
+
To resolve this, add argument `--forceversion 500`
|
| 52 |
+
|
| 53 |
+
This should be fixed in the next release of KoboldCpp, so if you are running a version later than 1.32 it should not be necessary.
|
| 54 |
+
|
| 55 |
## Repositories available
|
| 56 |
|
| 57 |
* [2, 3, 4, 5, 6 and 8-bit GGML models for CPU+GPU inference](https://huggingface.co/TheBloke/mpt-30B-chat-GGML)
|