InferenceIllusionist
commited on
Update README.md
Browse files
README.md
CHANGED
@@ -21,7 +21,7 @@ The challenge this time was placing more weight on Merlinite-7b as an unknown qu
|
|
21 |
* Measuring KL-divergences for new quantization types brought a deeper understanding of benchmarking and assessing model performance
|
22 |
* This signifcantly sped up the testing process by using MMLU as a base, narrowing down over 10 candidate linear merges to 1: merliniteX-blockB1
|
23 |
* Reaching the limitations of linear merging necessitated a pivot to reviewing the viability of SLERP, DARE-TIES, and Passthrough methods
|
24 |
-
* Thus a competing candidate merge pool was tested between different merge
|
25 |
* merliniteX-blockF2 (SLERP of Magic-Dolphin-7B and jaskier-7b-dpo in unorthadox proportions) was originally planned for release earlier this week
|
26 |
* Instead -blockB1 and -blockF2 were merged and the results were placed head to head in a final round of tests. Ultimately a more conventional execution of SLERP showed the best results for the final step.
|
27 |
|
|
|
21 |
* Measuring KL-divergences for new quantization types brought a deeper understanding of benchmarking and assessing model performance
|
22 |
* This signifcantly sped up the testing process by using MMLU as a base, narrowing down over 10 candidate linear merges to 1: merliniteX-blockB1
|
23 |
* Reaching the limitations of linear merging necessitated a pivot to reviewing the viability of SLERP, DARE-TIES, and Passthrough methods
|
24 |
+
* Thus a competing candidate merge pool was tested between different merge algorithms. Once more the list was narrowed from 10 candidates to 1: merliniteX-blockF2
|
25 |
* merliniteX-blockF2 (SLERP of Magic-Dolphin-7B and jaskier-7b-dpo in unorthadox proportions) was originally planned for release earlier this week
|
26 |
* Instead -blockB1 and -blockF2 were merged and the results were placed head to head in a final round of tests. Ultimately a more conventional execution of SLERP showed the best results for the final step.
|
27 |
|