Hugging Face's logo

ISTA-DASLab
/
c4ai-command-r-v01-AQLM-2Bit-1x16

Text Generation
Transformers
Safetensors
cohere
conversational
10languages
text-generation-inference
Inference Endpoints
Inference Endpoints
aqlm
Model card Files Community
Edit model card

Official AQLM quantization of CohereForAI/c4ai-command-r-v01 .

For this quantization, we used 1 codebook of 16 bits.

Results:

Model Quantization MMLU (5-shot) GSM8k (8-shot) Model size, Gb
CohereForAI/c4ai-command-r-v01 None 0.6755 0.6065 70.0
1x16 0.5719 0.3760 12.7
Downloads last month
82
Safetensors
Model size
6.36B params
Tensor type
FP16
·
I16
·
Inference API
Text Generation
Examples
Input a message to start chatting with ISTA-DASLab/c4ai-command-r-v01-AQLM-2Bit-1x16.
Model is too large to load in Inference API (serverless). To try the model, launch it on Inference Endpoints (dedicated) instead.

Collection including ISTA-DASLab/c4ai-command-r-v01-AQLM-2Bit-1x16