Hugging Face's logo

llmixer
/
Cat-Llama-3-70B-instruct-8.1bpw-h8-exl2

Text Generation
Transformers
PyTorch
llama
conversational
Inference Endpoints
text-generation-inference
8-bit precision
exl2
Model card Files Community
Edit model card

8.1 bpw exl2 quant of Cat-Llama-3-70B-instruct

More quants: https://huggingface.co/turboderp/Cat-Llama-3-70B-instruct-exl2

Downloads last month
7
Inference API
Text Generation
Examples
Input a message to start chatting with llmixer/Cat-Llama-3-70B-instruct-8.1bpw-h8-exl2.
Model is too large to load in Inference API (serverless). To try the model, launch it on Inference Endpoints (dedicated) instead.