TheBloke

/

ALMA-13B-Pretrain-AWQ

like 1

From the original model's README:

Note that ALMA-7B-Pretrain and ALMA-13B-Pretrain are NOT translation models. They only experience stage 1 monolingual fine-tuning (20B tokens for the 7B model and 12B tokens for the 13B model), and should be utilized in conjunction with their LoRA models for translation purposes.

Since it seems impossible to load original LoRA with AWQ quantized base model, the model is unusable for translation purpose.
Please, provide a quantizion for ALMA-13B instead.

See: https://discord.com/channels/1111983596572520458/1154744732807856231