Hugging Face's logo

Infermatic
/
Miquliz-120b-v2.0-FP8-dynamic

Text Generation
Transformers
Safetensors
llama
conversational
text-generation-inference
Inference Endpoints
fp8
Model card Files Community

Miquliz-120b-v2.0-FP8-dynamic

image/jpeg

This quant was made for infermatic.ai

Dynamic FP8 quant of Miquliz 120B v2.0 made with AutoFP8.

Model Details

  • Max Context: 32768 tokens
  • Layers: 140

Prompt template: Mistral 

<s>[INST] {prompt} [/INST]
Downloads last month
6
Safetensors
Model size
120B params
Tensor type
FP16
·
F8_E4M3
·
Inference Providers NEW
Text Generation
This model is not currently available via any of the supported third-party Inference Providers, and the model is not deployed on the HF Inference API.

Collection including Infermatic/Miquliz-120b-v2.0-FP8-dynamic