Hi Sail team! a request for data for quantization of your model
I sent a request at the web site, but I thought I should follow up here in case something goes awry.
Hello SEA AI Lab :) I am the author of https://github.com/robbiemu/llama-gguf-optimize , a library with a focus on preserving language diversity, the project leverages llama.cpp’s importance matrix approach to minimize quantization loss across distinct language domains. Bartowski is a major model quantization releaser for llama.cpp. These are names I imagine there is a chance someone at your project might know for their popularity. He and I have agreed to work together to create a custom imatrix for SAILOR2, using my toolkit. This work is hoped to produce the highest quality quantizations (at least in the regime of ~3bit quantization) available for your SAILOR2 model on popular platforms like Ollama and llama.cpp.
It would be best for us if we could draw on a sample of the data used to train the model: ensuring coverage of each language and especially any in-language coding samples used. If you cannot share even a part of the training data, that is fine, there are alternatives for us. But the thinking is you have done a better job of selecting appropriate samples than we will be able to. We are looking forward to hearing from you soon.
Happy new year!
Hi @robbiemu thank for your interest in Sailor2!
Please find more details in https://huggingface.co/sailor2 , including codebases and the statistics for your requirements.
Happy new year!