Model Fails to run on featherless.ai
#1
by
Darok-recursal
- opened
I dont know if it's the same as for your benchmarks, i suspect the model could be quantized and not specified. Anyone found out why
im so sorry for this late response! this was an earlier version of my model that is still currently being updated and i always make sure to release 4-bit medium and small gguf quants for each model release as these are the models i use personally!
Thanks for your response. Having an fp8 version sounds standard operation to me. If the model is quanted, it should be specified in the model name so it avoids any confusion. Could you provide an fp8 version? People on featherless want to try your model :D