nm-testing
/

opt-125m-fp8-dynamic

Text Generation

Inference Endpoints

text-generation-inference

Model card Files Files and versions Community

Edit model card

Made with https://github.com/neuralmagic/AutoFP8:

python quantize.py --model-id facebook/opt-125m --save-dir opt-125m-fp8-dynamic --activation-scheme dynamic

Downloads last month: 76

Safetensors

Model size

125M params

Tensor type

FP16

·

F8_E4M3

·