InferenceIllusionist commited on
Commit
2c53778
·
verified ·
1 Parent(s): 6a31197

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -2
README.md CHANGED
@@ -20,7 +20,7 @@ The challenge this time was placing more weight on Merlinite-7b as an unknown qu
20
  <b>Excalibur-7b</b> builds on past success and is the culimation of several learnings:
21
  * Measuring KL-divergences for new quantization types brought a deeper understanding of benchmarking and assessing model performance
22
  * This signifcantly sped up the testing process by using MMLU as a base, narrowing down over 10 candidate linear merges to 1: merliniteX-blockB1
23
- * Reaching the limitations of linear merging necessitated a pivot to reviewing the viability of SLERP, dares-ties, and passthrough methods
24
  * Thus a competing candidate merge pool was tested between different merge alogrithms. Once more the list was narrowed from 10 candidates to 1: merliniteX-blockF2
25
  * merliniteX-blockF2 (SLERP of Magic-Dolphin-7B and jaskier-7b-dpo in unorthadox proportions) was originally planned for release earlier this week
26
  * Instead -blockB1 and -blockF2 were merged and the results were placed head to head in a final round of tests. Ultimately a more conventional execution of SLERP showed the best results for the final step.
@@ -33,7 +33,7 @@ The challenge this time was placing more weight on Merlinite-7b as an unknown qu
33
 
34
  # Bonus Question - Vision Capabilities
35
 
36
- <b>Requires additional [mistral-7b-mmproj-v1.5-Q4_1.gguf](https://huggingface.co/koboldcpp/mmproj/tree/main) file for vision functionality)</b>
37
  <img src="https://i.imgur.com/4wbUrjf.jpeg" width="550"/>
38
 
39
 
 
20
  <b>Excalibur-7b</b> builds on past success and is the culimation of several learnings:
21
  * Measuring KL-divergences for new quantization types brought a deeper understanding of benchmarking and assessing model performance
22
  * This signifcantly sped up the testing process by using MMLU as a base, narrowing down over 10 candidate linear merges to 1: merliniteX-blockB1
23
+ * Reaching the limitations of linear merging necessitated a pivot to reviewing the viability of SLERP, DARE-TIES, and Passthrough methods
24
  * Thus a competing candidate merge pool was tested between different merge alogrithms. Once more the list was narrowed from 10 candidates to 1: merliniteX-blockF2
25
  * merliniteX-blockF2 (SLERP of Magic-Dolphin-7B and jaskier-7b-dpo in unorthadox proportions) was originally planned for release earlier this week
26
  * Instead -blockB1 and -blockF2 were merged and the results were placed head to head in a final round of tests. Ultimately a more conventional execution of SLERP showed the best results for the final step.
 
33
 
34
  # Bonus Question - Vision Capabilities
35
 
36
+ <b>Requires additional [mistral-7b-mmproj-v1.5-Q4_1.gguf](https://huggingface.co/koboldcpp/mmproj/tree/main) file for vision functionality</b>
37
  <img src="https://i.imgur.com/4wbUrjf.jpeg" width="550"/>
38
 
39