AetherArchitectural/GGUF-Quantization-Script

Mar 14

To generate the imatrix should you use wiki_test_data or any dataset, I don't see in your code you have used any dataset.

FantasiaFoundry

AetherArchitectural org Mar 14

•

edited Mar 14

@eramax

Your imatrix.txt is expected to be located inside the imatrix folder. I have already included a file that is considered a good starting option, this discussion is where it came from. (...)

The included data is Kalomaze's groups_merged.txt (30k).

Using wiki.train with 200*512 or 100*512 tokens seems also popular, although the quality compared to the "diverse data" example might not be substantially different/or could be worse. It's uncertain, you might need to test and do benchmarks in your use case.

FantasiaFoundry changed discussion status to closed Apr 19

AetherArchitectural
/

GGUF-Quantization-Script

imatrix test data?