InferenceIllusionist
commited on
Update README.md
Browse files
README.md
CHANGED
@@ -20,7 +20,7 @@ The challenge this time was placing more weight on Merlinite-7b as an unknown qu
|
|
20 |
<b>Excalibur-7b</b> builds on past success and is the culimation of several learnings:
|
21 |
* Measuring KL-divergences for new quantization types brought a deeper understanding of benchmarking and assessing model performance
|
22 |
* This signifcantly sped up the testing process by using MMLU as a base, narrowing down over 10 candidate linear merges to 1: merliniteX-blockB1
|
23 |
-
* Reaching the limitations of linear merging necessitated a pivot to reviewing the viability of SLERP,
|
24 |
* Thus a competing candidate merge pool was tested between different merge alogrithms. Once more the list was narrowed from 10 candidates to 1: merliniteX-blockF2
|
25 |
* merliniteX-blockF2 (SLERP of Magic-Dolphin-7B and jaskier-7b-dpo in unorthadox proportions) was originally planned for release earlier this week
|
26 |
* Instead -blockB1 and -blockF2 were merged and the results were placed head to head in a final round of tests. Ultimately a more conventional execution of SLERP showed the best results for the final step.
|
@@ -33,7 +33,7 @@ The challenge this time was placing more weight on Merlinite-7b as an unknown qu
|
|
33 |
|
34 |
# Bonus Question - Vision Capabilities
|
35 |
|
36 |
-
<b>Requires additional [mistral-7b-mmproj-v1.5-Q4_1.gguf](https://huggingface.co/koboldcpp/mmproj/tree/main) file for vision functionality
|
37 |
<img src="https://i.imgur.com/4wbUrjf.jpeg" width="550"/>
|
38 |
|
39 |
|
|
|
20 |
<b>Excalibur-7b</b> builds on past success and is the culimation of several learnings:
|
21 |
* Measuring KL-divergences for new quantization types brought a deeper understanding of benchmarking and assessing model performance
|
22 |
* This signifcantly sped up the testing process by using MMLU as a base, narrowing down over 10 candidate linear merges to 1: merliniteX-blockB1
|
23 |
+
* Reaching the limitations of linear merging necessitated a pivot to reviewing the viability of SLERP, DARE-TIES, and Passthrough methods
|
24 |
* Thus a competing candidate merge pool was tested between different merge alogrithms. Once more the list was narrowed from 10 candidates to 1: merliniteX-blockF2
|
25 |
* merliniteX-blockF2 (SLERP of Magic-Dolphin-7B and jaskier-7b-dpo in unorthadox proportions) was originally planned for release earlier this week
|
26 |
* Instead -blockB1 and -blockF2 were merged and the results were placed head to head in a final round of tests. Ultimately a more conventional execution of SLERP showed the best results for the final step.
|
|
|
33 |
|
34 |
# Bonus Question - Vision Capabilities
|
35 |
|
36 |
+
<b>Requires additional [mistral-7b-mmproj-v1.5-Q4_1.gguf](https://huggingface.co/koboldcpp/mmproj/tree/main) file for vision functionality</b>
|
37 |
<img src="https://i.imgur.com/4wbUrjf.jpeg" width="550"/>
|
38 |
|
39 |
|