nm-testing
/

opt-125m-fp8-dynamic

Text Generation

text-generation-inference

Inference Endpoints

Model card Files Files and versions Community

mgoin commited on Apr 27

Commit

0c6ac33

•

1 Parent(s): 8c5d6b6

Create README.md

Files changed (1) hide show

README.md +8 -0

README.md ADDED Viewed

	@@ -0,0 +1,8 @@

+---
+tags:
+- fp8
+---
+Made with https://github.com/neuralmagic/AutoFP8:
+```
+python quantize.py --model-id facebook/opt-125m --save-dir opt-125m-fp8-dynamic --activation-scheme dynamic
+```