GGUFs
Collection
I take requests, feel free to drop me a line in the community posts
•
40 items
•
Updated
•
2
Quantized from fp32 with love.
iMatrix .dat file was calculated using groups_merged.txt.
FP16 available here
An initial foray into the world of fine-tuning. The goal of this release was to amplify the quality of the original model's responses, in particular for vision use cases*
*Requires additional mmproj file. You have two options for vision functionality (available inside original repo or linked below):
Select the gguf file of your choice in Kobold as usual, then make sure to choose the mmproj file above in the LLaVA mmproj field of the model submenu:
Detailed results can be found here
Metric | Value |
---|---|
Avg. | 73.84 |
AI2 Reasoning Challenge (25-Shot) | 70.90 |
HellaSwag (10-Shot) | 87.93 |
MMLU (5-Shot) | 65.46 |
TruthfulQA (0-shot) | 70.82 |
Winogrande (5-shot) | 82.48 |
GSM8k (5-shot) | 65.43 |
Base model
InferenceIllusionist/Excalibur-7b