Update README.md

README.md

@@ -1,3 +1,5 @@
+# Initial Testing
+
 Some folks are claiming there's something funky going on with GGUF quanting for Llama 3 models.  I don't disagree.
 
 Some of those people are speculating that it has something to do with converting the raw weights from bf16 to fp16 instead
@@ -22,4 +24,32 @@ None specified:  size 29.9GB    PPL @ ???? 9.5158 +/- 0.15418    PPL @ Q4km 9.62
 As you can see, converting to fp32 has no meaningful effect on PPL compared to converting to fp16.  PPL is identical at full weight, 
 and the miniscule loss shown at Q4km is will within the margin of error.  There will no doubt be some people who will claim 
 "PpL iSn'T gOoD eNoUgH!!1!".  For those people, I have uploaded all GGUFs used in this test.  Feel free to use those files to do 
-more extensive testing on your own time.  I consider the matter resolved until somebody can conclusively demonstrate otherwise.
+more extensive testing on your own time.  I consider the matter resolved until somebody can conclusively demonstrate otherwise.
+
+
+# Continued Experiments 2024-05-11
+
+As an imatrix enjoyer, it has been bugging me whether the precision of the quant used to generate the imatrix actually 
+matters.  Scuttlebut says "yes, but only a little".  Logically, I don't think it should matter to a meaningful extent.  PPL 
+scales, so a value that is relatively important at fp16 should also register as relatively important at Q8 or even Q4.  
+
+To test this theory properly, I took failspy/Llama-3-8B-Instruct-abliterated and converted it to GGUF in both fp16 and fp32 
+formats.  I then quantized each of those GGUFs to both Q8_0 and Q4_0.  I then generated imatrices for each of those six 
+GGUFs.  Then I created eight GGUFs quantized at Q4_k_m:
+
+ - fp32 GGUF, fp32 imatrix
+ - fp16 GGUF, fp16 imatrix
+ - fp32 GGUF, fp32->Q8 imatrix
+ - fp16 GGUF, fp16->Q8 imatrix
+ - fp32 GGUF, fp32->Q4 imatrix
+ - fp16 GGUF, fp16->Q4 imatrix
+ - fp32 GGUF, no imatrix
+ - fp16 GGUF, no imatrix
+
+I ran PPL against all 8 quants, as well as the full fp16 and fp32 GGUFs.  Results:
+
+<in progress>
+
+Conclusion:
+
+<in progress>